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Foreword 


As  flight  control  systems  become  capable  of  providing  a  variety 
of  aircraft  response  types  and  aircraft  flight  envelopes  expand 
to  include  a  wider  range  of  angle  of  attack  and  speed,  the 
ability  to  predict  flying  qualities  becomes  increasingly 
difficult.  Traditional  parameters,  such  as  modal  characteristics 
and  time  delay,  cannot  totally  capture  the  relationship  of 
aircraft  dynamics,  task  performance  and  pilot  workload.  The 
success  of  Che  Handling  Qualities  During  Tracking  flight  test 
technique  led  to  the  thought  that  a  series  of  demonstration 
maneuvers  could  be  defined  for  a  variety  of  tasks  which  would 
augment  the  normal  aircraft  flying  qualities  description.  In 
order  to  be  useful,  such  maneuvers  must  be  well-defined  and 
suited  to  testing,  must  relate  to  the  operational  use  of  the 
vehicle  and  must  be  sensitive  to  parameters  used  in  the  design 
process. 

The  research  documented  in  this  four-volume  report  series  has 
developed  a  process  by  which  these  maneuvers  can  be  defined  and 
validated  as  well  as  an  initial  set  of  maneuvers  aimed  primarily 
et  agility  and  the  high-angle-of-attack  flight  regime.  A  key 
word  here  is  initial,  limited  resources  did  not  allow  this  effort 
to  address  all  aircraft  types  or  missions.  It  is  hoped  that  as 
various  agencies  and  companies  conduct  their  own  research,  they 
will  develop  additional  or  modified  maneuvers  and  add  them  to 
this  existing  set.  This  process  will  allow  the  maneuvers  to  keep 
pace  with  the  changes  in  aircraft  technology  and  operational 
missions  and  tasks.  New  maneuvers  should  be  sent  to  WL/FIGC_2, 
WPAFB  OH,  45433-7531.  An  updated  set  of  maneuvers  and  lessons 
learned  will  be  available  either  by  mail  or  electronically 
through  the  ARPANET  computer  network.  For  details,  contact  Tom 
Cord  at  (513)  255-8674.  The  resulting  maneuver  set  will  provide 
a  basis  from  which  demonstration  maneuvers  for  the  verification 
section  of  Mil-Std-1797B  can  be  defined. 
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Preface 


This  series  of  reports  proposes  aircraft  triaiieuvers  and  general  guidelines  foi  the  piloted 
evaluation  of  aircraft  flying  qualities  and  agility.  These  maneuvers  augment  rather  than  replace 
existing  fl>'ing  qualities  evaluation  techniques  and  arc  aimed  primarily  at  expanded  flight 
envelopes.  A  process  to  develop  new  evaluation  maneuvers  that  link  operational  requirements 
to  the  design  process  is  outlined  and  key  concepts  are  identified.  A  format  for  documenting 
and  selecting  useful  evaluation  maneuvers  is  also  described.  Finally,  the  evaluation  maneuvers 
and  data  demonstrating  their  sensitivity  to  design  parameter  variations  are  described. 

This  documentation  is  orgarrized  into  a  sequence  of  four  reports.  The  first  report,  subtitled 
"Maneuver  Development  Process  and  Initial  Maneuver  Set,"  includes  a  detailed  description  of 
the  research  conducted  as  well  as  a  summary  of  the  results.  It  describes  the  maneuver 
development  process  used  during  this  research  and  key  considerations  for  developing  new 
evaluation  maneuvers.  A  brief  summary  of  typical  results  observed  for  each  maneuver  tested  is 
also  included.  The  second  report,  subtitled  "Maneuver  Descriptions  and  Selection  Guide,"  is  a 
stand-alone  document  that  describes  the  maneuvers  tested  during  this  research.  It  documents 
the  ixif'-nt  of  each  maneuver,  the  aircraft  attributes  isolated,  the  techniques  required  to  fly  the 
maneuver,  as  well  as  presenting  a  cross  reference  to  help  select  the  most  valuable  maneuvers 
for  aircraft  evaluation.  The  second  repoit  is  the  beginning  of  a  standard  maneuver  reference 
guide  that  will  contain  a  wide  variety  of  evaluation  maneuvers  for  use  throughout  conflgiuation 
development  and  flight  test.  It  is  recommended  that  new  and  existing  evaluation  maneuvers  be 
added  to  this  report  to  prov  ide  a  source  of  evaluation  maneuvers  for  the  design  and  test 
community.  The  tliird  rc^^ort,  subtitled  "Simulation  Data,"  consists  of  detailed  information  on 
the  design  pa’-ameter  variations  tested,  subsequent  statistical  analyses  conducted  on  the 
simulation  data,  and  pilot  comments  and  ratirigs  from  tlie  testing.  The  fourth  report,  subtitled 
"Flight  Test  Plan,"  includes  a  preliminary  test  plan  for  tire  in-ilight  validation  of  tire  evaluation 
maneuvers. 
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Chapter  1 

Introduction  and  Summary  of  Results 
Background  and  Objectives 

Many  valuable  evaluation  maneuvers  currently  exist  for  heart-of-the-cnvelope  flying 
qualities  testing  such  as  the  well  established  "Handling  Qualities  During  Tracking"  (HQDT) 
techniques!  and  offset  landings.  However,  addidonal  maneuvers  are  needed  as  aircraft  flight 
envelopes  are  expanded  to  higher  angles  of  attack  and  as  aircraft  capabilities  axe  improved 
through  application  of  technologies  such  as  thrust  vectoring  and  forebody  vortex  controls.  As 
a  result,  this  efftxt  was  devised^  to  extend  HQDT  techniques  and  augment  current  evaluation 
methods  witli  new  maneuvers  specifically  designed  to  aid  in  the  evaluation  of  improved  aircraft 
capabilities  such  as  those  shown  in  Figure  1.  Such  maneuvers  would  be  used  to  identify 
deficiencies  while  an  aircraft  is  still  in  the  design,  development,  or  flight  test  stage  rather  than 
uncovering  problems  after  a  vehicle  has  entered  operational  use.  Tfiese  maneuvers  were  not 
developed  to  compare  an  aircraft  against  specification  parameters,  bat  instead  they  provide  a 
true  evaluation  of  the  flying  qualities  and  agility  of  an  aircraft  in  f  n  operationally  representative 
environment. 


Current  Evaluation  Maneuvers 


Extended  Envelope  Evaluation  Maneuvers 


Figure  1.  New  Maneuver*  for  the  Evaluation  of  Enhaixcedi  Aircraft  Capabliftlee 
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A  key  goal  during  the  development  of  these  maneuvers  was  to  establish  a  link  between 
operational  requirements  and  the  design  process.  This  is  necessary  to  ensure  that  the 
maneuvers  can  be  used  during  the  design  process,  Figure  2,  while  emulating  the  dynamic 
requirements  observed  in  an  operational  environment.  This  blends  operational  needs  back  into 
a  repeatable,  useful  evaluation  maneuver  similar  to  the  HQDT  techniques.  By  using  jin 
operarionally  relevant  evaluation  maneuver,  the  aircraft  design  can  be  evaluated  in  a  fashion 
more  like  it  will  be  used  by  the  pilots.  True  operational  relevance  is  somewhat  unlikely  for  a 
maneuver  that  is  intended  to  be  repeatable  and  provide  design  guidance.  However,  the 
Standard  Evaluation  Maneuver  Set  (S'fEMS)  maneuvers  are  designed  to  require  similar 
dynamic  lequirements  to  tliose  needed  during  operational  missions.  This  is  what  is  meant  by 
the  term  operationally  relevant  throughout  this  report  Tliis  research  was  not  intended  to  be  a 
criteria  development  effort  or  a  tactical  utility  stufly.  Instead,  a  sensitivity  between  each 
maneuver  and  various  design  parameters  was  established.  Therefore,  the  designer  now  has  an 
evaluation  tool  that  can  be  used  to  show  changes  in  aircraft  flight  characteristics  during  the 
development  phase.  Detailed  descriptions  of  the  evaluation  maneuvers  can  be  found  in  the 
second  report  of  this  report  scries.^  In  addition,  several  of  these  maneuvers  may  be  suitable 
for  the  development  of  design  criteria  or  tactical  utility  studies.  The  detailed  data  contained  in 
the  third  reporr  of  this  repon  .sequence  might  be  useful  as  a  starting  point  for  cither  of  these 
efforts, 

Another  objective  of  this  research  was  to  detine  an  effective  and  efficient  maneuver 
development  process  so  that  additional  maneuvers  could  be  generated  as  the  need  arises.  Such 
a  process  is  desirable  because  this  effort  could  not  define  a  complete  set  of  evaluation 
maneuvers.  Instead  it  documents  an  initial  set  of  standard  maneuvers  with  the  hope  that  other 
researchers  will  continue  to  add  useful  evaluation  maneuvers,  'fhe  maneuver  development 
process  and  key  concepts  used  during  this  contract  are  considered  to  be  necessary  to  provide 
consistently  high  quality  additional  maneuvers.  This  process  and  these  key  ideas  will  help 
keep  STEMS  a  "living"  document  as  new  maneuvers  are  added  for  new  technologies  or  to 
include  current  evaluation  methods. 

A  final  objective  of  this  effort  was  to  develop  a  preliminary  flight  test  plan.  The  flight  test 
plan  was  written  to  help  transition  the  experience  obtained  while  developing  tliese  maneuvers  in 
simulation  to  a  flight  test  validation  program.  It  is  written  generically  so  that  it  can  be  modified 
for  any  aircraft,  but  it  is  aimed  towards  aircraft  wiili  high  angle  of  attack  capability.  The  flight 
test  plan  will  be  suminarize<^  in  this  report  and  is  detailed  in  Reference  5. 
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Design  Parameter 
Modification 


StandarrJ  Evaluation  Maneuver  Set 

Afcraft  Aiiributes  hianeuYars 


Select  Maneuvers  Based 
on  Design  Characteristics 
and  Desired  Attributes 


Pitch  Authority  »  . Max  Pitch  Pull 

Roll  Auihcrity  V  Piicit  Angle  Capture 

Maneuverability  ^  Rolling  Defense 
Lon  Flying  Dual  Dual  Attack 

Lat  Rying  Oral  High  AOA  Lon  GA 
Axial  Flying  Qoal^V  High  AOA  Lat  GA 
Departure  Resist  ^  1g  Stab  Pushover 


Identify 

Deiiciencies 


Figure  2.  Evaluation  Maneuvera  TIa  Operational  Raquiramenta  to  Detlgn 
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Program  Structure 


A  swo-phase  program  was  used  to  achieve  the  above  objectives,  Figure  3.  The  first  phase 
was  an  initial  maneuver  development  phase  that  included  brainstorming  and  screening  efTorts. 
Botli  pilots  and  engineers  with  varying  backgrounds  worked  to  develop  a  large  set  of  potential 
maneuvers.  The  maneuvers  were  conceived  from  each  individual's  experience  and 
background.  Ihese  maneuvers  were  then  discussed,  refined,  and  sorted  into  a  more 
manageable  set  to  test  during  Phase  n.  No  flight  simulation  was  conducted  during  Phase  I. 


Survey  Literature 

Oevelop/Gather  Candidate  Maneuvers 
le'entiy  Evaluation  and  Design  Criteria 
Review  Team  Analysis  and  Screening 
of  Maneuvers 


Three  Simulations 
Define,  Test,  and  Validate  Maneuvers 
Confirm  UnK  Between  Maneuver,  Design 
Factors,  and  Operational  Environment 
Evaluate  Fligfrt  Test  Pian 


Final 
Maneuvers 


Flight 

Maneuver 

Development 

Process 


Figure  z.  Program  Structure 


The  second  phase  was  used  to  further  develop,  refine,  and  test  the  maneuvers  by  means  of 
piloted  flight  simulation  and  a  data  quality  review  process.  Phase  II  consisted  of  three 
simulations  with  periods  of  data  analysis  between.  Figure  4.  Tnis  structure  was  valuable 
because  it  provided  frv  periods  of  learning  between  the  simulations.  The  first  simulation  and 
subsequent  data  analysis  were  structured  to  be  somewhat  exploratory  to  inidslly  test  ideas  and 
techniques.  The  second  simulation  and  data  analysis  was  the  primary  data  gathering  effort  and 
the  third  simulation  was  used  to  answer  utirerolved  issues  and  conduct  additional  validation 
testing.  The  third  simulation  was  also  used  to  evaluate  and  refine  a  draft  flight  test  plan. 
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Phase  I  (- 
Potential  Cj/ 
Maneuvers 


Results 


Off-Site  Review 
Team  Analysis 
and  Critiques 


Flight  Test 
Engineers 


Results 


c>  Q  o  ^ 


Review  Team 
and  Critiques 


On-Site  Review 


Maneuver 


Maneuvers  Development 


Process 


Flight  Test 
Plan 


Figure  4.  STEMS  Three  Simulation  EMort 


A  recaning  feature  throughout  this  research  was  the  involvement  of  both  government  and 
industiy  personnel.  A  Review  Team  of  pilots  and  engineers  was  included  during  the  entire 
contract  to  evaluate  the  inancuvers  and  maneuver  development  process  as  well  as  provide 
guidance  for  remaining  work.  Many  members  of  the  Review  Team  also  participated  in  the 
simulation  efforts.  The  extensive  experience  and  diverse  background  anx>ng  the  Reviev/  Team 
members  was  very  beneficial  in  developing  and  reviewing  the  maneuvers.  Mots  with  both  test 
and  operational  cxicricncc  were  intentionally  included  so  that  the  maneuvers  could  benefit  ftom 
each  expertise.  Additionally,  the  Review  Team  engineers  had  varying  backgrounds  that 
covered  flying  qu;  lities.  agUity,  and  flight  test  experience.  Obviously  there  was  overlap  in  the 
Review  Team  members'  backgrounds,  but  each  had  valuable  unique  experience  to  offer. 
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Results 


Three  primary  products  were  developed  during  this  research:  a  process  to  develop  new 
maneuvers,  an  initial  set  of  evaluation  maneuvers,  and  guidelines  to  help  select  existing 
maneuvers,  Fig'ire  5.  The  maneuver  uc/elopment  process  can  be  used  to  produce  evaluation 
maneuvers  that  are  representative  of  operational  requirements  and  are  sensitive  to  design 
parameter  variations.  The  evaluation  maneuvers  are  valuable  during  design  and  flight  test  to 
evaluate  aircraft  attributes.  Finally,  the  maneuver  selection  guidelines  can  be  used  to  help  select 
the  most  important  maneuvers  for  the  given  test  objectives. 


Maneuver 

Selection 

Guidelines 


Guidelines  to  Help  User  Select 

Most  Beneficial  Maneuvers  for 

Specific  Test  Objectives 


Flight  Envelope 
Mission 
Configuration 
Attributes 


High  Quality, 

Well  Documented 
Evaluation  Maneuvers 


Tracking 
'  Acquisition 
Landing 


Figura  5.  Maneuver  Development  and  Selection 


There  aie  several  benefits  that  can  be  gained  through  the  use  of  these  tools.  High  quality 
evaluation  maneuvers  can  be  developed  more  efficiently.  The  time  required  to  test  an  aircraft 
can  be  reduced  by  using  predefined,  well -documented  evaluation  maneuvers.  The  time 
requii  ed  to  plan  for  a  test  can  be  reduced  and  the  quality  of  the  test  can  be  improved  by  using 
the  maneuver  selection  guidelines.  And  most  importantly,  a  mote  constnictivc  evaluation  can 
be  conducted  by  evaluating  key  aircraft  attributes. 
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The  maneuver  development  process  consists  of  several  key  elements  as  shown  in  Figure  6. 
One  of  the  most  important  ingredients  is  the  involvement  of  both  pilots  and  engineers 
throughout  the  development,  refinement,  and  analysis  of  each  maneuver.  It  is  also  beneficial  to 
draw  upon  a  variety  of  backgrounds  including  operational,  flying  qualities,  agility,  and  tlight 
test.  Ihe  operational  experience  is  especially  useful  to  help  be  the  maneuver  to  the 
requirements  of  the  final  user  -  the  operational  pilot.  The  flight  test  experience  is  invaluable  to 
help  define  repeatable,  measurable,  and  flyable  maneuvers.  Finally,  the  use  of  piloted  flight 
simulation  and  a  data  quality  review  process  were  also  key  to  efficiently  and  effectively 
defining  evaluation  maneuvers. 


Data  Quality 


Flight  Simulation  Review  Process 

Maneuver 

Development 

Process 

Figure  6.  Key  Elements  Required  to  Develop  Effective  Evaluation  Maneuvers 

Utilization  of  this  maneuver  development  process  resulted  in  the  identification  of  20 
maneuvers  that  can  be  used  to  evaluate  various  flying  qualities  and  agility  attributes  while 
maintaining  a  tie  to  operational  requirements.  Figure  7  shows  some  example  attributes  and 
operational  tasks  that  are  addressed  by  tliese  maneuvers.  These  20  evaluation  maneuvers  were 
shown  to  be  repeatable  and  provide  useful  data  for  the  design  process.  Descriptions  of  these 
maneuvers  have  been  developed  to  document  the  intent  of  each  maneuver,  the  techniques  used 
to  fly  the  maneuver,  and  potential  variations  to  the  maneuver.  These  maneuvers  are  primarily 


7 


designed  for  application  to  fighter  aircraft,  and  in  particular,  attributes  required  for  air-to-air 
combat  Many  of  the  maneuvers  were  developed  to  evaluate  extended  flight  envelope 
capabilities  in  terms  of  p)ost-stall/low-speed  maneuvering  because  of  relatively  recent 
improvements  in  aircraft  capability  in  this  flight  regime.  A  few  maneuvers  were  developed  for 
hean-of-the-envelope  operation  and  for  transport  class  aircraft  so  validate  this  concept  over  a 
wider  range  of  aircraft  classes  and  flight  phases. 


Air^fi 

Operational  Tasks 

*  Longitudinal  Flying  Qualities 

•  Guns  Tracking 

•  Lateral  Flying  Qualities 

•  Shift  Targets 

*  Directional  Flying  Qualities 

-  Turn  Reversal 

•  Axial  Flying  Qualities 

•  Weapons  Acquisition 

•  Multi-Axis  Flying  Qualities 

•  Nose  Intimidation 

•  Pitch  Authority 

•  Guns  Defense 

•  AOA  Authority 

•  Collision  Avoidance 

•  Roll  Authority 

•  Vertical  Lead  Turn 

•  Pitch  Control  Margin 

•  Missile  Jink 

•  Roll  Coordination 

•  SAM  Break 

•  Pitch  Performance 

•  Vertical  Reposition 

•  Roll  Performance 

•  Vertical  Attack 

•  Turn  Performance 

*  Attack  Abort  -  Bugout 

•  Axiai  Periormance 

•  Min  Time  Nose  High  Reversal 

•  Maneuverability 

•  Aerial  Refueling 

•  PIO  Tendencies 

‘  Formation  Flying 

<  Departure  Resistance 

*  Precision  Landing 

•  Frontside/Backside  Operation 

*  Side-Step  Approach  &  Landing 

Flnurs  7.  Example  Alrcnift  Attributes  and  Operational  Applications 


The  maneuvers  developed  under  this  research  range  in  complexity  and  character  as  shown 
in  Figure  8.  Some  maneuvers  tend  to  isolate  a  single  axis  while  others  are  multiple-axis  tasks. 
The  nature  of  the  maneuvers  varies  from  pure  open-loop  tasks  to  tight  closed-loop  tracking 
tasks.  And  the  pilot  technique  varies  from  sinictured  (technique  precisely  defined)  to 
unstructured  (freestyle  technique  allowed).  Because  of  the  wide  range  of  maneuvers  available, 
the  best  maneuver  to  use  for  an  evaluation  depends  upon  the  data  and  information  that  is  being 
sought.  Some  maneuvers  were  found  to  be  more  useful  for  qualitative  data  gathering  whereas 
others  were  much  better  suited  for  quantitative  analyses.  A  maneuver  selection  guide  was 
developed  to  help  the  user  select  potentially  useful  evaluation  maneuvers.  Once  the  user 
identifies  the  aircraft  characteristics  to  be  tested,  the  selection  guide  and  additional  iiifomiation 
provided  with  the  maneuver  descriptions  can  be  used  to  help  select  the  best  maneuvers. 
Descriptions  of  the  maneuvers  and  the  maneuver  selection  guide  are  included  Ln  Reference  3. 
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Figure  8.  General  Characteristics  of  the  initial  STEMS  Maneuvers 


The  maneuvers  sho\n'n  in  Figure  8  were  developed  to  allow  the  evaluation  of  a  range  of 
flying  qualities  and  agility  characteristics.  In  particular,  an  attempt  was  made  to  build  upon  the 
recent  agility  maneuver  research  conducted  in  References  6  and  T.  However,  the  maneuvers 
developed  under  tliis  contract  do  not  define  a  complete  set  of  evaluation  maneuvers.  These 
maneuvers  augment  existing  evaluation  maneuvers,  and  therefore  they  do  not  test  a 
comprehensive  set  of  aircraft  attributes.  A  much  widei  selection  of  maneuvers  would  be 
required  to  thoroughly  evaluate  an  aircraft.  Therefore,  it  is  recommended  that  existing 
evaluation  maneuvers  and  newly  developed  maneuvers  be  continually  added  to  STEMS  to 
increase  its  range  of  applicability.  Any  auditional  maneuvers  or  experiences  using  S  I'EMS 
should  be  forwarded  to  Wright  Iwaboratory/FIGC_2,  where  the  STEMS  maneuver  reference 
guide^  will  be  maintained  and  distributed. 
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Chapter  2 

Maneuver  Development  Process 


A  maneuver  development  process  was  exercised  and  refined  during  this  research.  This 
maneuver  development  process  is  recommended  as  an  effective  and  efficient  method  to  develop 
aircraft  maneuvers.  This  chapter  summarizes  the  process,  lessons  learned,  and  key  elements 
required  to  develop  rdgh  quality  evaluation  maneuvers.  Chapter  4  will  document  how  this 
maneuver  development  process  evolved  and  how  20  evaluation  maneuvers  were  developed  and 
tested  during  lire  lefiiiemerit  of  this  process. 

Maneuver  Development  Process  Overview 

Tlie  maneuver  development  process  is  summarized  in  Figuie  9  and  will  be  described  in 
detail  in  the  following  sections.  Prior  to  beginning  the  development  of  a  new  maneuver,  details 
of  the  aircraft  characteristics,  such  as  the  flight  envelope,  mission,  configuration,  and  other 
attributes,  must  be  specified  so  that  the  resulting  maneuver  will  meet  the  te.st  objectives.  The 
first  step  in  the  maneuver  development  process  is  to  define  further  these  aircraft  characteristics 
and  propose  candidate  maneuvers  that  might  meet  the  test  objectives,  'i'hese  candidate 
maneuvers  are  then  screened  to  identify  the  most  promising  ones.  The  next  step  utilizes 
simulation  to  further  develop  and  refine  the  candidate  maneuvers.  A  data  quality  review  is 
incorporated  into  this  step  to  ensure  that  the  maneuver  produces  sufficient  data  to  be  useful 
during  design.  The  final  step  is  used  to  validate  the  maneuver  through  additional  simulation 
and  in-flight  testing.  Also,  a  very  important  part  of  the  last  step  is  the  doc'imentation  of  the 
final  evaluation  maneuver.  Anyone  who  develops  or  uses  an  evaluation  maneuver  is 
encouraged  to  document  their  results  to  Wright  Laboratory/FIGC_2  for  incorporation  into 
STEMS. 


Candidate  Maneuver  Definition  and  Screening 

The  first  step  in  the  maneuver  development  process  is  designed  to  explore  a  range  of 
potential  maneu\ ers  and  select  the  most  promising  ones  for  furtiier  development.  This  requires 
the  involvement  of  both  engineers  and  pilots  with  design,  flight  test,  operational,  flying 
qualities,  and  agility  experience  as  indicated  in  Figure  10.  A  variety  of  backgrounds  is 
beneficia’  to  help  provide  a  wider  selection  of  potential  maneuvers  and  concepts. 
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'  Flight  Envelop^ 
'  Mission 
'  Configuration 
'  Attributes 


Candidate  Maneuver 
Definition  and  Screening 


Maneuver  Oevelopmenr  and 
Refirtement  Using  Simulation 


Figure  9.  Maneuver  Development  Process 


Engineers 


Pitots 


Figure  10.  Engineers  and  Pilots  Are  Needed  to  Develop  Evaluation  Maneuvers 
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The  aircraft  characteristics  to  be  evaluated  must  be  defined  Tirst.  Specific  attributes,  such  as 
roll  authority  or  Pilot  Induced  Oscillation  (PIO)  tendencies,  should  be  identified  to  help 
evaluate  if  the  data  generated  from  the  maneuver  isolates  the  characteristics  of  interest.  Figure 
1 1  shows  a  list  of  attributes  that  adequately  encompass  the  initial  set  of  evaluation  maneuvers. 
This  list  of  attributes  should  grow  as  additionai  maneuvers  are  added  to  STEMS.  Potential 
operational  scenarios  that  require  the  desired  attributes  should  also  be  identified  at  this  time. 
Some  example  operational  scenarios  are  listed  in  Figure  12.  A  stronger  tie  to  operational 
requirements  can  be  made  if  operational  scenarios  are  considered  throughout  maneuver 
development  and  refinement.  Initial  maneuver  concepts  can  then  be  developed  from  these 
operational  .scenarios  and  desired  test  attributes  by  using  "brainstorming",  literature  searches, 
and  experience.  At  this  stage,  the  maneuvers  may  be  very  "sketchy"  and  several  maneuver 
concepts  should  be  proposed. 


•  Longitudinal  Flying  Qualities 

♦  Pitch  Performance 

•  Lateral  Flying  Qualities 

*  Roll  Performance 

•  Directional  Flying  Qualities 

•  Turn  Performance 

•  Axial  Flying  Qualities 

♦  Axial  Performance 

•  Muiti-Axis  Flying  Qualities 

•  Maneuverability 

•  Pitch  Authority 

•  Energy  Maneuverability 

•  Roll  Authority 

•  PIO  Tendencies 

•  Pitch  Control  Margin 

♦  oepartijie  Resistance 

•  Roll  Coordination 

•  Frontside/Bacl-.side  Operation 

Figure  11.  Example  Aircraft  Attributes 


•  Guns  Tracking 

•  SAM  Break 

•  Shift  Targets 

•  Vertical  Reposition 

•  Turn  Reversal 

•  Vertical  Attack 

•  Weapons  Acquisition 

•  Attack  Abort  -  Bugout 

•  Nose  Intimidation 

•  Min  Time  Nose  Fligh  Reversal 

•  Guns  Defense 

♦  Aerial  Refueling 

•  Collision  Avoidance 

•  Formation  Flying 

•  Vertical  Lead  Turn 

•  Precision  Landing 

•  Missile  Jink 

•  Side-Step  Approach  &  Landing 

Figure  12.  Example  Operational  Scenarios 


A  screening  process  is  then  conducted  to  identify  strong  candidate  maneuvers  for  further 
development  and  refinement.  One  of  the  key  goals  of  the  maneuver  development  process  is  to 
maintain  a  link  between  design  and  operation,  so  this  link  should  be  evaluated  in  the  screening 
process.  Figure  1 3  is  a  form  that  summarizes  some  of  the  key  questions  that  should  be 
considered.  A  qualitative  evaluation  must  be  performed  at  this  point  because  the  maneuvers 
have  not  been  fully  defined  and  simulation  data  is  not  yet  available.  However,  the  maneuver's 
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potential  for  operational  relevance  and  its  suitability  for  design  anu  engincenr.g  use  should  be 
considered.  The  applicability  of  the  maneuver  throughout  the  development  piocess  can  also  be 
factoreo  into  the  screening  process.  A  maneuver  is  more  valuable  if  it  car  br  used  from  early 
developmental  simulations  through  to  flight  test.  Finally,  potential  problems  and  issues  such 
as  human  factors  considerations  or  requiiements  for  specialized  displays  should  be  identified. 
These  issues  may  be  strong  enough  to  eliminate  a  maneuver  from  further  consideration  or  may 
simply  indicate  the  need  to  explore  certain  issues  during  simulation. 


Operatiunal  Relevance: 

1.  Arc  dynamic  conditions  and  pilot  activity  representative  of  operational  use? 

2.  Does  the  maneuver  require  use  of  an  extended  tlighi  tnveiope? 

3.  Is  the  maneuver  useful  for  evaluation  of  current  aircraft? 

4.  Is  the  ma-tcuvcr  useful  for  evaluation  of  future  aircraft?  6 


5  4  3  2  1 


5.  Link  to  operational  use 


Strong  [  J  IJ  L  j  r  ] ;  .1  [,. 


Yes  No 

□  i'.l 

□  IT] 

n  (~ 
r:]  □ 

Weak 


Suitability  for  Design/Pmgineoring  Use: 

6.  Is  the  maneuver  mcaningful/imcrprcublc  in  pajamctcis  mscful  lo  designers? 

7.  Docs  the  maneuver  produce  data  of  sufficient  magnitude  to  guide  modificaiicn  of  ilic  design? 


Yes  No 


8.  Can  die  m.ancuvtr  be  uscii  to  produce  pilot  opinion  ratings  (such  as  CHR)? 


6  5  4  3 


1 


9.  Evaluolion  of  flying  qualitic.s 

10.  Evaluation  of  agihty 

1 1 .  Qualitative  data  generated 

12.  QuaniUutivc  data  generaleu 

13.  Pilot  comment  data 


Strong J 1  ;  J .  ! !  1  Weak 

Strong  n  i  ~  "  IJ  Ij  ( “  Weak 
High  [ J  L I IJ  ili  Lj  [_]  None 
High  LJ .  J  ;  J  „  [J  None 

Strongly  Tied  to  Design  Process  □  t  j  ’  J  I’]  t.l  None 


Additional  Concern.s: 

14.  Is  die  maneuver  well  dermed  and  rcpcuiabic? 

15.  Can  cntry/cxit  conditions  be  readily  established? 

16.  Display.s  rcquircd/pilot  cues  necessary 

17.  Difficulty  to  Hy  niaiicuvcr 


5  4 

Conventional  ,  - 1  . J 
Easy  I  '  i' : 


3  2 


Yes  No 

I"'  : 

,  ::i 

.  .  Unique 
.  i  Difficult 


Conimcnt.s: 


Please  Circle  Your  Overall  Rating  of  This  Maneuver: 

1.  Strong  evaluation  maneuver,  definitely  p.in  of  die  final  sci  of  maneuvers. 

2.  Possibly  good  maneuver,  needs  additional  testing  or  refincnicni. 

3.  Inconclusive,  inodificatioiis  or  additional  testing  required. 

4.  Possibly  poor  maneuver,  results  curremly  inconclusive  but  not  promising. 

5.  Poor  evaluati'.ei  niancuvcr,  do  not  coniiiiiie  testing. 


Figure  13,  Maneuver  Evaluation  Form 


All  of  the  candidate  maneuvers  should  be  examined  using  the  above  considerations.  The 
candidate  maneuvers  that  most  closely  match  the  test  objectives  can  then  be  singled  out  for 
further  development  in  simulation.  However,  even  good  candidate  maneuvers  may  have  some 
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weak  points.  The  quality  of  die  final  evaluation  iraneuver  can  be  improved  by  working  on 
these  weak  points  during  simulation  development  of  the  maneuver. 


Maneuver  Development  and  Refinement  Using  Simulation 

The  development  and  refinement  of  a  candidate  maneuver  requires  periods  of  simulation, 
data  analysis,  and  maneuver  evaluation  as  diagrammed  in  Figure  14.  The  maneuver  concepts 
developed  during  the  Candidate  Maneuver  Definition  and  Screening  process  tend  to  be  vaguely 
defined  and  need  to  be  further  developed  during  simulation.  Fhis  is  initiated  by  formulating 
more  precise  maneuver  descriptions  from  the  initial  concepts  and  defining  the  data  to  be 
collected.  A  nominal  set  of  aircraft  dynamics  can  be  selected  foi  die  maneuver  refinement 
process.  The  specific  aircraft  dynamics  chosen  are  not  critical;  however,  they  should  be 
representative  of  the  test  aircraft  Pilots  and  engineers  can  then  refine  the  maneuver  by  quickly 
trying  various  techniques  during  simulation.  The  objectives  of  tlie  maneuver  refinement  are  to 
increase  the  flyabiiity  and  repeatability,  improve  the  quality  of  data  generated,  and  enhance  the 
operational  relevance  of  the  maneuver.  Additionally,  Cooper-Harper  Rating*  performance 
criteria  should  be  developed  for  flying  qualities  evaluation  maneuvers. 


Figure  14.  Phase  It  Maneuver  Development  Process 


Variations  in  design  parameters  are  tested  after  the  maneuver  has  been  sufficiently  refined 
a'.d  developed.  Design  parameters  that  are  pertinent  to  the  attributes  being  tested  should  be 
selected  and  appropriately-sized  variations  should  be  defined.  These  variations  must  be  large 
enough  that  a  resulting  change  can  reasonably  be  expected.  The  variations  can  be  based  on 
MIL-STD-1797A^  and  other  design  guidance  and  should  be  large  enough  that  they  could  be 
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expected  to  alter  the  flying  qualities  from  Level  I  to  Level  2  or  from  Level  2  to  Level  3.  A  test 
matrix  can  be  developed  aftci  choosing  the  design  parameters  and  their  ranges  of  variation.  A 
full  factorial  matrix  (test  every  combination)  or  mwe  efficient  Design  of  Experiments^  (DOE) 
or  TaguchiS  S  metliods  car.  be  used  to  develop  the  test  matrix.  The  design  parameter  variations 
are  then  evaluated  during  simulation  using  a  blind  test  that  includes  at  least  two  pilots. 

After  gathering  simulation  data,  the  maneuver  is  evaluated  for  its  ability  to  generate  reliable 
information  for  design  guidance  and  its  ability  to  isolate  the  desired  te.«t  attributes.  Time 
history  data,  pilot  comments,  and  pilot  ratings  (when  appropriate)  should  be  examined  for  theii^ 
sensitivity  to  design  parameter  variations  and  to  pilot  variability.  The  time  history  data  can  be 
used  either  in  a  raw  form  to  compare  pilot  inputs,  control  surface  activity,  and  aircraft  states,  or 
it  can  be  processed  into  numerical  measure  of  merits  such  as  maximum  roll  rate  or  time  to 
capture.  All  of  the  simulation  data  is  dien  used  to  detennine  if  the  maneuver  can  be  used  to 
reliably  identify  design  deficiencies.  The  simulation  data  should  exhibit  a  sensitivity  to  design 
parameter  variations  and  an  insensitivity  to  pilot  variability  in  order  for  the  maneuver  to  be  used 
with  confidence  during  the  design  process.  The  data  quality  can  be  examined  in  various  forms 
such  as  statistical  analyses  of  measure  of  merit  data,  graphical  display  of  time  history  and  pilot 
rating  data,  and  comparisons  of  pilot  comments. 

Quantitative  and/or  qualitative  data  may  'ot  obtained  witli  a  maneuver.  The  ability  to 
generate  quantitative  data  can  be  gauged  by  the  quality  of  measure  of  merit  data  obtained  with 
the  mane  iver.  Measures  of  merit,  such  as  time  to  capture  or  n.aximum  pitch  rate,  can  be 
calculated  from  tlie  simulation  time  history  data  Statistical  techniques  are  then  used  to  evaluate 
the  amount  of  variability  in  the  data  and  thereby  judge  the  quality  of  quantitative  data  generated 
from  the  maneuver.  The  change  in  each  measure  of  merit  that  is  due  to  a  design  parameter 
variation  should  be  coinpaicd  to  the  change  due  to  pilot  variability.  11'  the  measures  of  merit  are 
much  more  sensitive  to  design  parameter  changes  than  to  differences  in  pilots,  then  quantitative 
data  can  be  used  foi  design  guidance.  Various  methods  are  available  to  perform  -his  analysis 
and  a  detailed  example  of  one  technique  is  mcluded  m  Qiapter  4. 

Additional  qualitative  data,  such  as  pilot  comments,  pilot  ratirigs*^  and  answers  to 
questionnaires,  should  be  gathered  during  simulation  and  considered  for  their  value  to  the 
design  process.  An  example  pilot  comment  card  that  is  helpful  to  evaluate  the  candidate 
maneuver  is  shown  in  Figure  15.  The  ai.'swcrs  to  these  questions  and  the  pilot  evaluations 
during  the  design  parameter  testing  should  be  reviewed  for  sensitivity  to  design  parameter 
variations  aiid  insensitivity  to  pilot  variability  just  like  the  numerical  measure  of  merit  data.  If 
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the  pilot  comments  and  ratings  successfully  correlate  to  the  design  pararne'er  variations,  then 
the  maneuver  can  be  considered  to  generate  useful  qualitative  data  for  design  guidance.  Tiie 
results  of  the  quantitative  and  qualitative  data  analysis  should  be  documented  with  the 
evaluation  maneuver  so  that  other  users  of  the  maneuver  will  know  what  type  of  data  they  can 
anticipate  to  gatficr  with  the  maneuver. 


1.  How  well  does  the  maneuve'’  represent  the 
operational  task  element? 

5  4  3  2  1 

Closely  [  ]  n  "  :  j  Remotely 

2.  Is  the  maneuver  well  defined?  Please 
describe  any  specific  techniques  used. 

5  4  3  2  1 

Well  Defined  ;_j  □  □  □  i  J  Poorly  Defined 

3.  Is  the  maneuver  repeatab'e  and  easy  to  fly? 

BepolS 

4.  Did  variations  in  design  parameters  result  m 
observable  differences  in  response? 

5  4  3  2  1  Not 

c  'i  1  i  i-  -■  '  '  Signit'cantly 

^  Different 

5.  Would  entry/e;  it  conditions  be  difficult  to 
establish  during  fight  test? 

5  4  3  2  1 

Easy  □  [,  j  ;  J  IJ  u'U  Impossible 

6.  What  information  i$  required  (e.g.  airspeed, 
bank  angle,  target  aircraft,  etc.)? 

Conventional  -n  . 

information  ■  Specialized 

Displays 

7.  Additional  comments: 

Figure  15.  Maneuver  Summary  Commeni  Card  Used  During  Simulations 


Potential  human  factors  considerations  can  also  be  estimated  at  this  lime.  Combinations  of 
time  history  data  analysis  and  responses  to  pilot  questionnaires,  such  as  shown  in  Figure  16, 
can  be  used  to  assess  maneuvers  that  aie  potetitially  disorienting  or  have  the  potential  for  g 
Induced  loss  Of  Consciousness  (GLOC).  Mathematical  models  tliat  predict  the  pilot's 
susceptibility  to  GL  OC  or  spatial  disorientation  based  on  aircraft  acceleration  time  history 
prof'les  can  be  used  at  this  time  to  evaluate  the  maneuver.  Potentially  dangerous  maneuvers 
should  be  noted  at  this  time  arid  either  refined  or  carefully  tested  in  a  motion-base  simulation  or 
in  flight. 
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PREFLIGHT  QUESTIONNAIRE 

1 .  How  many  hours  of  sleep  did  you  get  last  night? 

2.  Your  last  meal  prior  to  this  flight  was. ..(circle  one) 

breakfast 

lunch 

dinner 

snack 

How  long  has  it  been  since  you  ate  the  above  meal? 

3.  Your  health  today;  (circle  one) 

Excellent  -  no  health  problems 
Fair  -  slight  head  cold/allergies 
Poor  -  severe  head  cold/flu 


POSTFLIGHT  QUESTIONNAIRE 

1 .  Did  you  experience:  During  what  part  of  the  maneuver? 

A  tumbling  sensation? 

Lightheadedness? 

Motion  sickness? 

Disorientation? 

Other? 

2.  Do  you  feel  there  is  a  potential  for  spatial  disorientation  during  this  maneuver? 

If  yes,  what  aspect(s)  of  the  maneuver  contribute  to  this? 


If  yes,  do  you  feel  that  training  couid  be  done  to  minimize  this  potential? 


Figure  16.  Human  Factors  Questionnaire  Used  During  the  Simulations 


All  of  the  simulation  data  and  analyses  should  be  reviewed  to  detennine  the  nent  action  to  be 
taken  on  the  maneuver.  As  Figure  14  indicated,  the  maneuver  can  be  accepted  as  ready  for 
use,  refined  further  during  simulation,  or  discarded.  An  evaluation  fOTm  such  as  was  shown  in 
Figure  13  can  be  used  to  help  assess  the  overall  quality  of  the  maneuver  to  determine  if  it  has 
been  developed  sufficiently.  Additional  simulation  refinement  of  the  maneuver  may  prove 
valuable  if  identified  weaknesses  can  be  corrected. 
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Maneuver  Validation  and  Documentation 


The  maneuver  can  be  used  as  a  design  and  evaluation  tool  once  the  Maneuver  Development 
and  Refinement  process  is  complete;  however,  additional  validation  and  documentation  of  the 
maneuve:  is  highly  recommended.  For  instance,  the  range  of  applicability  of  the  maneuver 
should  be  checked.  This  validation  can  be  conducted  by  testing  the  maneuver  with  different 
aiicraft  to  make  sure  that  the  maneuver  is  not  unique  to  the  aircaft  that  was  used  to  develop  it. 
The  maneuver  should  be  generic  enough  that  it  can  be  altered  slighdy  for  a  specific  aircraft  or 
test.  Also,  analytical  and  subjective  assessments  of  potential  motion  effects  can  be  conducted, 
particularly  if  fixed-base  simulation  testing  is  utilized  for  the  maneuver  developmenL 
Maneuvers  that  exhibit  potential  warning  signs  may  need  additional  testing  in  a  motion-base 
simulator  with  appropriate  dynaniic  capability.*^  And  of  course,  the  final  validation  of  a 
iiianeuver  comes  through  in-flight  testing.  Some  of  the  initial  STEMS  maneuvers  have  been 
successfully  tested  as  pait  of  Air  Force  Test  Pilot  School  projects*'***5  and  efforts  are  being 
made  to  validate  all  the  maneuvers  through  various  flight  test  programs.  Several  issues  can  be 
more  closely  examined  during  a  flight  test  validation  program  such  as  safety  of  flight  issues, 
ability  to  set  up  and  execute  the  maneuver,  and  data  requirements  for  tlie  maneuver. 

Documentation  of  the  final  evaluation  maneuver  is  critical  to  the  usefulness  of  STEMS. 
Newly  developed  nianeuvers  should  be  sent  to  Wriglit  Laboratory/FIGC_2  for  inclusion  into 
STEMS^  so  that  it  becomes  a  "living  document."  The  intent  is  that  STEMS  wili  eventually 
serve  as  a  comprehensive  guide  to  evaluation  maneuvers.  Each  evaluation  maneuver  .should  be 
documented  on  a  standard  lorm  as  schematically  shown  in  Figure  17.  Additional  background 
information  on  the  maneuver,  lessons  learned,  and  validation  testing  conducted  with  the 
maneuver  can  be  included  in  nairaiivi';  pages  that  accompaiiy  the  maneuver  description  page. 
Also,  an  "electronic"  appendix  to  the  STEMS  maneuver  reference  guide  has  been  developed  to 
include  example  trajectories  for  die  maneuvers.  These  examples  can  be  viewed  on  the  AGHJE- 
VU  flight  najectory  visualization  program*^  to  assist  in  visualization  of  U.e  maneuver  and  how 
it  is  to  be  performed.  The  example  AGILE- VU  files  are  also  being  maintained  and  distributed 
from  Wright  Laboratory/I'1GC_2. 

The  various  sections  of  the  maneuver  description  form  document  tiie  reasons  why  the 
maneuver  would  be  flown,  what  type  of  attributes  it  measures,  what  mission  it  is  intended  to 
represent,  how  to  set  up  and  fly  the  maneuver,  guidelines  on  developing  Coeper-Harper  Rating 
performance  criteria,  imponant  notes  and  comments  about  the  maneuver,  as  w'ell  as  potentiLal 
variations  to  the  maneuver.  A  narrauve  description  also  accompanies  each  maneuver  to 
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document  addidonal  ioformation  not  found  <hi  the  brief  maneuver  description  fomi. 

Maneuvers  should  be  written  somewhat  generally  so  tliey  can  be  tailored  to  suit  specific  test 
objectives.  They  also  may  be  modified  based  on  configuraticm  dependent  placards,  safety  of 
flight  issues,  or  unique  capabilities.  Additionally,  specific  setups  may  need  to  be  altered  based 
on  the  test  aircraft  (and  target  aircraft,  if  one  is  called  for)  perfonnance  capabilities.  The 
maneuver  descriptions  include  representative  test  conditions  such  as  airspeed,  altitude,  and 
Angle  Of  Attack  (AOA),  but  the  specific  conditions  to  be  tested  are  left  to  the  evaluator. 
Multiple  variations  of  the  maneuver  are  also  briefly  described  to  show  potentially  useful 
alternative  approaches  such  as  testing  throttle  setting  variations  for  configurations  with  thrust 
vectoring. 
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Maneuver  Development  Lessons  Learned 


Several  valuable  lessons  were  learned  while  evolving  the  maneuver  development  process 
described  above.  Some  of  these  lessons  will  be  summarized  here  to  improve  the  quality  of 
new  evaluation  maneuvers  and  minimize  the  time  required  to  develop  them.  First,  it  was 
extremely  beneficial  to  include  both  pilots  and  engineers  throughout  the  maneuver  development 
and  analysis  process.  Second,  it  was  valuable  to  include  pilots  with  operational  as  well  as  test 
experience.  Tliird,  it  is  critical  to  evaluate  the  quality  of  data  obtained  from  a  maneuver  prior  to 
using  it  for  the  evaluation  of  an  aircraft  design.  Fourth,  it  was  found  that  piloted  flight 
simulation  could  be  used  to  efficiently  and  effectively  develop  maneuvers.  Finally,  DOE  test 
techniques  were  used  extensively  during  this  research  and  some  observations  and 
considerations  are  summarized  in  this  chapter. 

Pilot  and  Engineer  Involvement 

It  is  recomiTiended  that  both  pUots  and  engineers  be  involved  throughout  the  development  of 
new  evaluation  maneuvers.  It  is  also  advantageous  to  include  more  than  one  pilot  and  more 
than  one  engineer  to  benefit  from  additional  viewpoints  and  experiences.  Iheie  may  be  some 
overlap  and  blend  of  knowledge  between  the  pilots  and  engineers,  but  each  tends  to  have  a 
specialized  background  that  can  improve  the  value  of  the  maneuver.  In  general,  engineers  were 
needed  to  determine  the  constraints  on  the  maneuver  and  the  data  obtained  from  the  maneuver. 
They  could  identify  the  type  of  data  needed  for  design  g  uidance  such  as  singlc*axis  flying 
qualities  information,  control  hannony  evaluations,  maximum  j>erformance  meai,urements, 
control  margin  validations,  departure  resistance  testing,  and  others.  They  also  suggested 
constraints  on  how  the  data  should  be  generated  such  as  requiring  full  stick  inputs  for 
maximum  performance  or  allowing  "freestyle”  pilot  inputs.  Engineers  also  defined  the 
important  parameters  for  the  initial  conditions  such  as  requiring  an  initially  stabilized  AOA. 
zero  pitch  rate,  or  a  constant  g  turn.  The  pilots  were  invaluable  in  maintaining  operationally 
representative  conditions  and  defining  techniques  for  the  maneuvers.  They  also  had  important 
suggestions  which  improved  the  flyability  and  repeatability  of  the  maneuvers.  Pilot  experience 
was  important  in  identifying  critical  mission  sclents  and  modifying  the  maneuvers  to  better 
represent  those  conditions. 


Benefits  of  Operational  and  Test  Experience 


The  influence  of  pilots  with  operational  and  test  experience  on  the  maneuver  development 
process  is  also  importanL  The  operational  experience  is  important  because  of  the  desire  to  link 
these  evaluation  maneuvers  to  operational  requirements.  Pilots  with  operational  experience 
have  a  good  understanding  of  how  the  aircraft  will  really  be  used  in  training  and  q)eration  and 
have  experience  in  tactics  and  techniques.  Flight  test  experience  is  also  important  to  incTease 
the  overall  quality  of  the  maneuver  and  data  acquired  from  it  Flight  test  skills  help  provide 
insight  as  to  how  to  improve  the  maneuver  setups  and  execution  for  better  repeatability  and  data 
quality.  Pilots  with  flight  test  backgrounds  also  have  a  better  awareness  of  safety-of-flight 
issues  for  maneuvers  that  are  intended  to  be  used  during  flight  test  And  Anally,  flight  test 
pilots  can  help  the  maneuver  development  process  because  they  also  tend  to  have  a  good 
understtiiJing  of  flight  dynamics  and  tlie  Cooper-Harper  Rating  process. 

During  this  research,  both  "test"  and  "operational"  pilots  were  included  to  be  sure  to  include 
both  specialties.  It  was  found  that  only  one  operational  pilot  and  one  test  pilot  were  needed  at 
any  one  time  to  support  the  maneuver  development  and  data  gathering  process.  In  actuality 
many  pilots,  including  tliosc  participating  in  this  study,  have  a  good  blend  of  both  skills.  As  a 
result,  it  may  be  beneficial  to  have  both  an  "operational"  and  a  "test"  pilot  involved,  but  having 
separate  "test"  and  "operational"  pilots  may  not  be  necessary  if  tlie  pilots  involved  have  a  good 
blend  of  experience. 

Data  Quality  Review 

The  final  kc>  element  in  defining  maneuvers  for  design  is  the  requirement  for  a  maneuver 
review  process.  It  is  important  to  review  the  data  generated  from  a  maneuver  ana  have  an 
understanding  of  its  sensitivity  to  design  parameter  variations  before  using  iL  It  may  be 
misleading  to  develop  a  maneuver  with  a  single  set  of  aircraft  dynamics  and  then  use  it  to 
evaluate  design  modifications  or  other  aircraft,  llie  data  repeatability  must  be  checked  to 
evaluate  the  amount  of  pilot  variability  expected  in  the  data.  The  pilot  variability  should  then  be 
compared  to  the  changes  observed  due  to  design  parameter  variations  to  determine  which 
pieces  of  data  can  be  used  in  tire  design  process.  This  data  review  should  be  applied  to  both 
quantitative  and  qualitative  data.  Ihis  implies  that  data  is  required  from  at  least  two  pilots 
before  any  strong  conclusions  can  be  made  about  tire  maneuver.  The  research  conducted  under 
this  contract  used  a  formal  Review  Team  with  a  variety  of  backgrounds  to  judge  the  value  of  a 
maneuver.  The  Review  Team  approach  was  efficient  for  the  simultaneous  develtqrment  of 
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several  maneuvers;  however,  a  much  simplified  review  process  could  be  used  when 
developing  a  few  maneuvers. 

Use  of  Piloted  Flight  Simulation 

Flight  simulation  was  found  to  be  an  effective  tool  in  the  developmf:nt  and  evaluation  of  the 
maneuvers.  Different  approaches  and  techniques  to  fly  a  maneuver  could  be  tried  quickly  and 
eliminated  from  consideiation  by  using  flight  simulation.  Additionally,  a  quick  appreciation 
could  be  gained  for  the  type  of  data  generated  and  the  aircraft  characteristics  evaluated.  The 
maneuver  can  then  be  refined,  while  still  in  simulation,  to  produce  better  quality  data.  The 
simulation  effort  under  this  research  was  divided  into  three  simulation  entries.  This  multiple 
simulation  approach  was  valuable  because  it  allowed  for  periods  of  data  review  between 
simulations.  The  maneuvers,  and  test  techniques,  could  then  be  refined  during  the  next 
simulation  based  on  previous  data. 

The  Phase  I  portion  of  this  research  was  used  to  generate  several  concepts  and  potential 
evaluation  maneuver;  however,  no  simulation  was  included.  As  a  result,  the  maneuvers  were 
not  ready  to  be  used.  Effort  was  spent  discussing  and  refining  the  maneuvers  rather  than 
developing  them  for  use.  In  contrast,  the  Phase  II  simulations  were  found  to  be  much  more 
effective  for  exposing  the  advantages  and  disadvantages  of  a  maneuver.  Different  approaches 
and  techniques  to  fly  a  maneuver  could  be  tried  quickly  and  eliminated  fixim  consideration  by 
using  flight  simulation.  Additionally  a  quick  appreciation  could  be  gained  for  the  type  of  data 
generated  and  the  aircraft  characteristics  evaluated.  The  background  work  and  preparations 
conducted  during  Phase  I  certainly  aided  maneuver  development  during  the  simulations  but,  in 
retrospect,  more  tinK  should  have  been  spent  flying  the  maneuvers  and  less  time  should  be 
spent  discussing  them. 

Observalions  from  DOE  Testing 

Maiiy  of  the  simulation  test  matrict:;  used  during  this  research  were  based  on  Design  of 
Experiment  techniques.  Fractional  factorial  matrices  were  used  to  minimize  the  data 
requirements  so  that  as  many  maneuvers  as  possible  could  be  developed.  The  EXDE  techniques 
worked  well  far  quantitative  data,  but  it  was  diflicult  to  analyze  the  qualitative  data  since 
multiple  design  parameters  were  being  simultaneously  varied.  In  general,  the  fractional 
factorial  tests  appear  to  be  appropriate  for  maneuvers  that  generated  numerical  data  that  is 
amenable  to  statistical  analyses,  but  they  arc  not  genaally  recommended  for  maneuvers 
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designed  to  gatlier  pilot  comments.  Also,  it  can  be  difficult  to  efficiently  expand  testing  after  an 
initial  data  set  is  taken.  Simple  lest  matrices  could  be  augmented  to  include  additional  design 
parameters,  but  more  complex  matrices  could  not  be  augmented.  As  a  result,  it  is  very  valuable 
to  perform  a  quick,  qualitative  check  of  the  intended  test  matrix  prior  to  gathering  a  complete 
data  set  In  general,  the  DOE  techniques  evaluated  during  tliis  research  can  be  valuable  for  a 
screening  effort  but  do  not  appear  to  be  appropriate  for  a  criteria  development  effort. 
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Chapter  3 

Simulation  Setup  and  Test  Techniques 


Tlie  simulation  hardware  and  aircraft  models  that  were  utilized  during  the  three  STEMS 
simulations  are  desoibed  in  this  chapter.  Additional  details  on  the  test  setup  including  the 
design  parameters  selected,  the  test  matrix  selection  techniques  used,  and  data  gathering 
procedures  utilized  are  also  included. 


Simulation  Setup 


All  of  the  simulation  testing  was  conducted  in  a  fixed-base,  40  ft  diameter  simulation  dome 
as  shown  scl.ematically  in  Figure  18.  This  dome  contains  an  F-15C  cockpit  and  controllers 
including  a  conventional  center  stick  with  characteristics  as  shown  in  Figure  19.  This 
ci  iwstaiion  was  used  for  both  fighter  and  transport  aircraft  testing.  The  only  modification 
m-'ut-.''  v/hen  testing  the  transport  aircraft  was  that  the  military  power  detente  was  renxjvtd  from 
the  thioiilc  hardware  so  that  the  full  throttle  range  could  be  used.  The  simulation  setup  utilized 
a  General  Electric  Compuscene  computer  graphics  imaging  system  for  out-the-window  visual 

TT^.ai  /^<\tv»rtncr*Ano  ct/cfArrt  n  v'AAnAmr  H«2toKoiCA  tKot  iv>nrACAntAH  Air 

Force  Base.  A  video  projection  system  and  model  boxes  were  used  to  display  F- 15  targets  for 
air-to-air  tasks.  Additionally,  a  Compuscene  generated  KC-10  model  was  used  for  the 
refueling  probe  tracking  task.  Standard  F-15C  Head-Up  Display  (HUD)  hardware  was  used  to 
display  modified  F-15E  HUD  symbology  to  the  pilot  as  shown  in  Figure  20.  Several  of  the 
features  on  the  HUD  could  be  altered  to  test  various  Cooper-Harper  Rating  performance 
criteria.  Tlie  size  and  depression  of  the  reticle  could  be  altered  easily,  and  horizontal  and/or 
vertical  error  bars  could  be  displayed  to  help  the  evaluation  pilot  concentrate  on  the  errois  in  a 
single  axis.  For  closed-loop  tasks,  shoot  cue  lights  on  the  cockpit  canopy  bow  were 
programmed  to  illuminate  when  the  pilot  had  achieved  a  capture  for  the  desired  length  of  time. 
A  dual  processor  Gould  SEL  computer  was  used  to  drive  tlie  simulation.  The  aircraft  model 
and  most  crewstation  input/output  ran  at  a  60  Hz  update  rate.  Some  secondary  displays  were 
updated  at  a  slower  rate,  but  the  primary  displays  and  the  controller  inputs  were  sampled  at  60 
Hz,  The  total  simulation  time  delay  from  pilot  input  to  visual  scene  update  was  estimated  to  be 
between  94  msec  and  111  msec  for  the  simulation  setup  used  in  this  testing.*  ^ 


40  ft,  Fixed-Base  Dome 


GE  Compuscene  Visual 
Scene  and  Target  Display  - 


F-15C  Cockpit  and  Controllers 


Video  and  Laser  Target  Projectors 


FIguro  18.  Simulation  Dome  Uaed  for  Testing 


Three  primary  aircraft  tncxiels  were  used  during  these  simulations.  Two  models  were 
simplified  generic  aircraft  and  the  third  was  a  complex,  fully  nonlinear  aircraft  model.  The 
generic  models  represented  a  fighter  aircraft  and  a  transport  aircj-aft.  The  generic  fighter  model 
was  used  for  the  majority  of  the  maneuver  develqtment  and  design  parameter  testing.  This 
model  was  based  on  the  McDonnell  Douglas  Aerospace  (MDA)  Generic  Aircraft  (GENAIR) 
simulation  tool.  GENAIR  provided  the  capability  to  vary  the  closed-loop  aircraft 
characteristics  easily  a.nd  efficiently  over  a  wide  range  while  maintaining  realistic  nonlinear 
performance  as  illustrated  in  Figure  21.  This  simulation  model  has  been  used  successfully  in 
the  past  to  develop  high  AOA  maneuvers  for  flight  test^®,  to  develop  low  and  high  AOA  flying 
qualities  criteria'^-20,  and  conduct  agility  evaluarions^i.  The  F-15  STOL  and  Maneuvering 
Technology  Demonstrator  (S/MTD)  Multi-System  Integrated  Controls  (MuSiC)  aircraft  model 
was  used  to  validate  the  maneuvers  developed  with  GENAIR.  The  MuSIC  model  is  a 
complex,  high-fidelity  simulation  model  that  includes  a  nonlinear  database  and  a  complete 
control  system^^.  The  MuSIC  model  was  used  to  validate  the  flyability  of  the  maneuvers  with 
a  complex  model  that  had  significantly  different  performance  and  flying  qualities  than  the 
GENAIR  nxKlel. 
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The  generic  fighter  model  was  used  for  the  majority  of  the  testing.  Its  baseline  performance 
was  representative  of  a  modem  high-performance  fighter  airci'aft;  however,  its  dynamic 
response  could  be  changed  to  test  design  parameter  variations.  The  dynamic  response  was 
completely  generic  and  varied  wddely  during  testing  to  emulate  variations  in  closed-loop  aiicraft 
dynamics.  The  design  parameters  varied  during  these  simulations  r re  listed  in  Figure  22.  The 
generic  aircraft  model  could  be  used  to  vai^  many  more  design  parameters  than  shown  on  this 
figure,  but  these  were  sufficient  to  evaluate  the  maneuvers  developed  in  this  research.  Several 
longitudinal  command  types  were  used  during  the  generic  fighter  testing.  These  included  AOA 
command,  AOA  rate  xjmmand,  load  factor  command,  and  pitch  rate  command  systems.  Most 
of  the  testing  was  conducted  with  AOA  and  load  factor  command  systems,  but  specific 
maneuvers  were  also  validated  with  the  rate  command  systems.  A  stability  axis  roll  rate 
command  system  was  implemented  in  the  lateral  axis  so  that  the  pilot  commanded  a  coordinated 
roll  through  lateral  stick  uputs.  A  sideslip  command  system  was  implemented  in  the 
directional  axis  and  a  limited  amount  of  roll  rate  resulted  from  rudder  pedal  inputs. 


Altbreviation  Descfiation 


AOAMAX 

CAP 

CLMAX 

CMOTYP 

DCG 

LALPHA 

LATDYN 

LONDYN 

LONSHP 

LONSNS 

MALPHA 

PMAX 

PDLIM 

TAUENG 

TIMDEL 

TR 

TV 

TW 

ZSP 

ZW 

WSP 


Maximum  AOA  (low  speed)  and  maximum  load  factor  (high  speed) 

Control  Anticipation  Parameter  (related  to  short  period  frequency) 

Maximum  lift  coefficient 

Longitudinal  command  type;  AOA.  AOA  rate,  pilch  rate,  or  load  factor 

Center  of  gravity  location 

Lift  curve  slope  and  pitch  rate  lead  term 

Combination  of  lateral  dynamics  (TR.  PMAX,  etc.) 

Combination  of  longitudinal  dynamics  (CAP,  ZSP,  AOAMAX,  LONSHP,  etc.) 
Whether  or  not  nonlinear  longitudinal  stick  shaping  was  used 
Longitudinal  stick  sensitivity 
Longitudinal  stability 

Maximum  attainable  stability  axis  roll  rate 
Roll  acceleration  limiter 
Engine  lime  constant 

Pure  time  delay  added  to  the  simulation  (in  addition  to  inherent  time  delay) 

Roll  mooe  time  constant 

Whether  or  not  thrust  vectoring  was  used 

Multiplier  on  the  baseline  thrusl-to-woight  ratio 

Short  period  damping 

Inverse  of  the  pitch  rate  time  constant  for  firsLorder  pitch  rate  command 
Short  period  frequency 


Figure  22.  Design  Parameters  Varied  During  the  Generic  Fighter  Testing 


The  generic  transport  model  was  used  for  a  minimal  amount  testing  fo  determine  if  the 
STEMS  maneu  vers  could  be  applied  to  aircraft  other  than  fighters.  The  baseline  performance 
and  dynamics  of  this  model  were  representative  of  a  responsive  powered-lift  transport  alreraft. 
However,  the  dynannic  respionse  of  the  oanspon  model  was  varied  during  the  simulation 
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testing.  Figure  23  shows  the  design  parameters  that  were  varied  during  the  transport 
maneuvers.  This  is  a  much  abbreviated  list  compared  to  the  design  parameters  varied  during 
the  fighter  testing  because  of  the  limited  amount  of  simulation  time  devoted  to  transport  testiug. 
An  AOA  command  system  was  implemented  in  the  longitudinal  axis;  a  roll  rate  command 
system  was  used  in  the  teral  axis;  and  a  sideslip  command  system  was  implemented  in  the 
directional  axis. 


Abbreviation  Cesciiplica 


CAP 

PMAX 

TR 

ZSP 

WSP 


Control  Anticipation  Parameter  (related  to  short  period  frequency) 

Maximum  attainable  stability  axis  roll  rate 

Roll  mode  time  constant 

Short  period  damping 

Short  period  frequency 


Figure  23.  Design  Parameters  Varied  During  the  Generic  Transport  Testing 


The  MuSlC  aircraft  model  was  used  for  a  ;najoiity  of  the  maneuver  validation  and  for  some 
design  parameter  varia  i i  n  testing.  This  model  is  a  built  upon  the  F- 1 5  S/MTD  database  with 
modifications  that  repn  ..  ;nt  the  addition  of  axisymmethc  pitch  and  yaw  vectoring  nozzles.  The 
MuSIC  aircraft  mode!,  !■  ^j,re  24,  was  developed  under  another  effort  to  investigute  the  tactical 
utility  of  pitch  and  yaw  v  noring  during  air  combat  engagemenLs22.  That  study  resulted  in  the 
development  of  new  hig^i  ^OA  fighter  tactics^^^  Some  of  these  tactics  were  incorporated  into 
the  maneuvers  develope  1  luring  the  STEMS  research.  The  MuSIC  aircraft  was  flown  in  two 
modes:  Post-Stall  (PST) ;  n  and  PST  off.  When  PST  is  engaged,  the  MuSIC  aircraft  is  an 
extremely  agile  configura  on  with  essentially  no  AOA  limit  and  very  good  high  AOA  roll 
authority.  When  PST  is  c  T,  the  MuSIC  model  has  greatly  reduced  roll  authority  at  moderate  to 
high  AOA  and  can  only  re  ich  approximately  40*  AOA.  The  variation  between  PST  on  and 
PST  off  was  used  as  an  additional  evaluation  for  some  of  tiie  STEMS  maneuvers.  Tlie  MuSIC 
model  was  also  flown  in  conjunction  with  the  Fighter  Airframe  Propulsion  Integration 
Predesign  (FAPIP)  piogram.24  During  this  testing,  STEMS  maneuvers  were  used  to  evaluate 
nozzle  design  variations  including  maximum  nozzle  rate  capabilities  and  various  nozzle  time 
delays.  This  provided  further  validation  of  the  ability  to  use  STEMS  maneuvers  diuing  the 
design  process. 


30 


Axisym  metric 


A  primary  objective  of  this  research  was  to  tie  evaluation  maneuvers  to  the  design  process 
so  that  they  can  be  used  to  improve  a  design  prior  to  flight  test  or  operational  use.  To  verify 
this  link,  design  parameters  were  varied  during  the  simulations  and  data  was  gathered  to 
evaluate  the  ability  to  use  a  maneuver  during  the  design  pixxtcss.  High-level,  augmented 
design  parameters  were  used  during  this  testing  to  maintain  generic  results  rather  than 
configuration  dependent  trends.  In  other  words,  design  pai  ameters  such  as  augmented  short 
period  frequency  were  tested  rather  than  static  margin,  pitch  control  power,  and  control  system 
gains.  The  augmented  (aircraft  plus  control  system,  closed-loop  dynamics)  design  parameteis 
tested  were  similai  to  those  resulting  from  an  equivalent  system  analysis.^-^  The  justification  of 
this  test  approach  lies  in  the  fact  that  with  modern  control  systems,  the  pilot  is  evaluating  the 
overall  system  response  instead  of  basic  stability  derivatives  and  bare  airframe  response.  The 
achievable  aircraft  dynamics  depend  upon  several  basic  parameters  such  as  the  center  of  gravity 
location,  wing  design,  leading  edge  extension  shape,  and  available  control  powers.  However, 
the  trade-offs  between  these  basic  design  considerations  and  tlie  achievable  dynamics  is  very 
configuration  dependent  and  beyond  the  scope  of  this  study.  Additionally,  depending  upon  the 
control  powers  available,  a  wide  range  of  dynamic  responses  can  be  achievul.^^  As  a  result, 
variations  in  bare  airframe  design  parameters  were  not  tested.  Instead,  equivalent  sy.stcm  type 
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parameters  weit;  varied  to  establish  if  a  maneuver  could  be  i’  sd  to  uncover  design  deficiencies 
that  affect  the  pilot's  ability  to  complete  the  mission  or  task. 

The  particular  design  parameters  selected  for  testing  and  the  ranges  over  which  they  were 
varied  depended  upon  the  maneuver.  Only  a  few  parameters  could  be  tested  for  each  maneuver 
because  of  the  number  of  maneuvers  being  developed.  It  was  believed  ic  be  sufficient  to 
establish  a  sensitivity  to  two  or  three  design  parameters  for  a  particular  maneuver.  The  Review 
Team  identified  a  potential  list  of  design  parameters  for  each  maneuver  during  Phase  I. 
Engineering  judgment  was  then  used  to  select  a  few  of  the  most  likely  parameters  to  aiTcct  each 
maneuver.  The  range  for  each  design  parameter  was  ba.sed  on  existing  criteria,  recent  research, 
and/or  engineering  judgment.  Aii  attempt  was  made  to  select  large  enough  ranges  such  that  a 
good  maneuver  would  show  a  difference  between  configurations.  For  example,  short  period 
frequency  might  be  varied  between  a  MIL-STD-1797A  Level  1  value  to  a  Level  2  value.^  The 
design  parameters  being  tested  for  a  particular  maneuver  were  sometimes  changed  between 
simulations  if  the  simulation  data  analysis  showed  linlc  or  no  effect  due  to  a  design  parameter. 
A  new  range  of  variation  was  tested  if  the  Review  Team  suggested  that  the  original  range  was 
too  small  or  too  large. 


Desigr  of  Experiment  Test  Approach 

A  Design  Of  Experiments^®  test  approach  was  used  during  simulations  to  test  the  sensitivity 
of  the  maneuver  to  design  parameter  variations.  Th;s  statistically-based  test  approach  allows 
several  design  parameters  to  be  tested  with  a  minimal  number  of  configurations.  Post¬ 
simulation  data  analysis  then  relies  on  statistical  techniques  to  isolate  the  effects  of  each  design 
parameter  on  the  data  being  evaluated.  Tliesc  statistical  analy.scs  were  conducted  for  each 
candidate  measure  of  merit  foi  each  maneuver.  The  outcome  of  the  statistical  tests  indicates  the 
change  in  the  measure  of  merit  that  is  due  to  each  design  parameter  and  provides  a  measure  of 
statistical  confidence  in  the  answer.  These  analyses  also  provide  an  indication  of  the  amount  of 
pilot  variability  in  the  simulation  data. 

The  DOE  test  technique  was  adopted  as  a  design  parameter  screening  approach  for  the 
maneuver  development  process.  During  the  maneuver  development,  it  was  necessary  to  test 
each  maneuver's  sensitivity  to  several  design  parameters.  An  efficient  technique  was  needed  to 
help  identify  which  design  parameters  could  be  used  to  altei  the  performance  of  the  aircraft 
during  that  particular  maneuver.  The  DOE  approach  enables  the  use  of  much  smaller  test 
matrices  —  typically  half  the  size  of  staitdard  tests  (w  even  smaller.  These  reduced  matrices 
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were  sufficient  to  establish  sensitivities,  but  may  not  have  been  thorough  enough  to  define 
criteria  boundaries  or  establish  recommended  values  for  individual  design  parameters. 
However,  this  research  was  not  intended  to  be  a  criteria  development  effort. 


Tlie  IX)E  techniques  require  specific  test  matrices  be  used  so  that  the  statistical  analyses  Ciin 
be  conducted  properly.  A  variety  of  test  maai  is  have  been  designed  and  can  be  selected 
depending  on  the  number  of  parameters  that  the  researcher  is  invesdgating.^0  The  test  matrix 
most  commonly  used  during  this  maneuver  development  research  allowed  three  design 
parameters  to  be  tested  at  two  values  each  while  requiring  only  four  configurations.  If  all 
possible  combinations  were  tested,  then  eight  configurations,  and  therefore  twice  the  data, 
would  have  been  required.  The  DOE  test  matrix  is  also  referred  to  as  a  fractional  factorial  as 
opposed  to  a  full  factorial  matrix  (all  combinations  tested).  Another  test  approach  could  have 
been  used  in  which  one  configuration  would  have  been  selected  as  a  baseline,  and  then  each  of 
the  three  design  parameters  could  have  been  individually  varied  to  generate  a  total  of  four 
configiuations.  It  was  reasoned  that  this  approach  would  result  in  a  more  "local"  indicator  of 
trends,  whereas  the  DOE  approach  would  measure  rrore  of  a  "global”  sensitivity  and  would 
allow  more  comprehensive  statistical  tests.  The  specific  combinations  of  design  parameter 
values  that  needed  to  be  iested,  accoiding  to  DOE  guidelines,  are  shown  in  Figure  25  fora 
three  i'actor  (design  parameter)  test.  Statistical  analyses  can  be  conducted  to  isolate  design 
parameters  A,  B,  and  C  if  data  is  available  for  the  four  highlighted  configurations.  These 
statistical  tests  indicate  the  difference  in  average  measure  of  merit  values  between  faces  of  the 
test  cube  as  diagrammed  on  the  lower  portion  of  this  figure.  Larger  test  matrices  were  also 
used  that  allowed  as  many  as  seven  design  parameters  to  be  evaluated  while  using  only  eight 
test  configurations.  Figure  26  summarizes  the  three  DOE  test  matrices  used  during  this  study. 
The  positive  (+)  and  negative  (-)  signs  indicate  the  two  values  for  each  design  parameter. 


Ihe  DOE  techniques  had  advantages  and  disadvantages  for  this  particular  application. 

ITiese  matrices  required  fewer  configurations  to  be  tested  when  several  design  parameters  were 
being  evaluated;  therefore,  it  allowed  more  maneuvers  to  be  de\  eloped  in  the  limited  amount  of 
time  available.  It  worked  favorably  for  the  numerical  analysis  of  measure  of  merit  data  but  it 
made  the  qualitative  analysis  more  difficult  and  less  conclusive.  Tlie  nunerical  analysis  was 
simplified  because  standard  statistical  techniques  could  be  applied  to  process  tlie  data 
However,  qualitative  data  analysis  is  usually  performed  by  making  comparisons  between 
configurations  where  only  one  design  parameter  is  being  altered.  Unfortunately,  the  statistical 
techniques  used  to  process  the  numerical  data  could  not  be  used  on  pilot  con unents.  As  a 
result,  it  was  difficult  for  the  Review  Team  to  isolate  the  effects  of  a  single  design  parameter 
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because  when  they  compared  pilot  comments  between  configurations  tliere  were  always  at  least 
two  design  parameters  varying.  This  made  it  difficult  to  determine  which  comments  should  be 


attributed  to  which  design  parameters.  For  example,  some  similar  deficiencies  may  be 


observed  when  comparing  a  configuration  with  a  lov/  short  period  frequency  and  low  damping 
to  a  configui  ation  with  high  short  period  frequency  and  high  damping.  Some  comments  may 
be  traced  directly  to  a  single  design  parameter,  but  others  are  hard  to  identify  because  both  a 
low  frequency  and  a  high  damping  have  the  effect  of  slowing  the  aircraft  response  to  pilot 
input.  The  qualitative  data  analysis  became  increasingly  difficult  as  the  number  of  design 
parameters  was  increased.  The  seven  design  parameter  matrix  shown  in  Figure  26  was  poorly 


suited  for  the  review  of  pilot  comments  because  so  many  parariteters  were  being  varied 


simultaneously. 


Configurations  Evaluated 
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Figure  25.  Frectlonal  Factorial  Approach  Used  to  Screen  Design  Parameters 
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(Note:  Same  Matrix  as 
Described  in  Figure  25) 


Figure  26.  Design  of  Experimems  Test  Matrices  Used  During  Simulations 

It  was  also  somewhat  difficult  to  add  new  parameters  to  a  DOE  rest  matrix  during  a 
simulation  or  between  simulations.  Post-simulation  statistical  analysis  must  be  conducted 
before  any  results  can  be  obtained,  consequently,  results  obtained  during  a  simulation  cannot 
be  used  to  alter  tlie  test  l  aatrix  during  that  simulation.  Therefore,  it  is  valuable  to  perform  a 
quick  qualitative  evaluation  of  the  test  matrix  to  help  refine  it  prior  to  collecting  data.  Some 
DOE  test  matrices  were  successfully  augmented  to  add  iui  additional  design  parameter  after  data 
had  been  acquired,  bur  in  other  cases  an  entirely  new  test  matrix  was  nxjuired.  Tne  simple 
three  factor  test  mao  ix  described  in  Figure  25  was  easy  to  augment  with  additional  design 
parameters  as  shown  in  Figure  27.  Unfortunately  this  technique  did  not  allow  a  direct 
statistical  comparison  between  all  four  design  parameters  shown  on  this  figure,  but  it  did  allow 
previously  collected  data  to  be  the  reu.sed.  The  larger,  n»ore  complex  DOE  matrices  v/cre 
difficult  or  impossible  to  augment  in  a  similar  manner.  So,  a  completely  new  matrix  was  tested 
if  a  change  was  required  between  simulations. 
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Figure  27.  Three  Factor  DOE  Matrix  Augntented  for  Additional  Parameters 
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Care  must  be  used  when  choosing  the  design  parameters  and  matrix  fca"  DOE  testing.  In 
particular,  the  test  factors  should  be  ''orthogonal"  (not  dependent  upon  any  other  test  factor) 
and  cannot  be  "confounded"  (interdependent)  for  the  statistical  techniques  to  be  valid.  This 
was  inadvenently  violated  with  a  test  matrix  during  the  first  simulation.  Three  design 
variations  were  conducted  during  this  test  Short  period  frequency,  short  period  damping,  and 
nonlinear  longitudinal  command  shaping  were  tested.  Unfortunately,  the  longitudinal 
command  shaping  was  implemented  by  a  method  that  resulted  in  confounding,  therefore 
voiding  the  results.  The  command  shaping  was  used  to  jjrovide  a  fast  response  when  a  large 
difference  between  the  commanded  and  actual  AOA  existed,  and  it  slowed  the  response  for 
small  stick  inputs.  It  was  implemented  by  scheduling  short  period  frequency  and  damping 
with  the  command  error  so  that  both  a  desired  acquisition  and  a  desired  tracking  response  could 
be  obtained.  How'ever,  the  statistical  analyses  were  invalid  because  of  the  interdependencies 
among  command  shaping,  short  period  frequency,  and  short  period  damping. 

The  statistics  associated  with  this  DOE  process  were  valuable  to  those  with  experience  or  a 
background  in  statistics,  but  were  confusing  to  other  members  cf  the  Review  Team.  The 
Review  Team  members  had  mixed  success  interpreting  the  raw  statistical  r^ata  because  of  their 
varying  levels  of  statistical  training,  Those  members  with  good  statistical  backgrounds  were 
able  to  interpret  the  raw  statistics  effectively.  Other  Review  Team  members  had  much  better 
results  in  reviewing  processed,  summary  statistical  information.  The  final  fomnat  used  to 
summarize  the  statistics  will  be  discussed  briefly  in  Chapter  5  and  more  fully  detailed  in 
Reference  4. 


Data  Taking  Procedures 

A  set  of  well-defined  data  gathering  procedures  was  followed  during  the  simulations  to 
ensure  consistent  data  quality.  Guidelines,  such  as  the  number  of  pilots  who  evaluated  the  test 
matrices,  the  number  of  repeats  flown,  and  the  methods  used  to  conduct  Cooper-Harper  Rating 
evaluations  were  established  during  the  first  simulation.  It  was  especially  imponant  to  follow 
consistent  data  gathering  procedures  for  the  data  that  was  to  be  statistically  processed.  Data 
was  gatliered  from  a  miiiimum  of  two  pilots  so  that  the  potential  for  pilot  variability  could  be 
studied.  When  possible,  testing  was  conducted  so  that  data  from  one  test  pilot  and  one 
operational  pilot  would  be  available  foi  analysis.  This  helped  maximize  the  type  and  variety  of 
comments  and  evaluations  available  from  a  maneuver.  In  the  end,  it  is  believed  to  be  imponant 
to  include  data  i  mrn  a  test  pilot  and  an  operational  pilot,  but  this  may  not  be  necessary  if  the 
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pilots  participating  have  a  btoad  range  of  expeiience.  Data  was  gatliered  from  four  pilots  in  a 
few  maneuvers  to  investigate  pilot  variability  further. 

Each  pilot  flew  the  maneuver  with  a  set  of  baseline  dynamics  until  he  was  familiar  with  the 
setup  and  any  specific  techniques  required.  Next  the  pilot  was  given  a  set  of  dynamics  for 
©valuation.  He  flew  the  configuration  until  he  had  established  a  relatively  consistent, 
comfortable,  and  operationally  representative  technique.  Three  data  runs  were  then  recorded 
for  post-simulation  analysis.  Additional  data  runs  would  have  been  beneficial,  but  only  three 
were  used  to  minimize  the  time  required  and  amount  of  data  processing  needed  for  each 
maneuver. 

Pilot  comments  were  continuously  recorded  during  the  simulations.  The  comments  were 
transcribed  after  the  simulations  and  were  edited  for  clarity  and  brevity.  The  resulting  pilot 
comments  have  been  sorted  by  maneuver  and  configura  Jon  and  are  included  in  Reference  4. 
These  comments  were  important  to  help  evaluate  the  qualitative  data  generated  from  eacii 
maneuver.  In  particular,  the  pilots  were  asked  to  comment  on  the  configuration  response  and 
their  ability  to  perform  the  task.  A  simulation  comment  card  was  completed  by  the  pilots  after 
completing  all  of  the  configuration  evaluations.  This  questionnaire  was  intended  to  capture  the 
pi)  't  s  overall  opinion  of  the  maneuver.  The  final  questionnaire  was  showTi  in  Chapter  2, 
Figure  15.  The  pilots  completed  written  comment  cards  during  the  first  simulation  and  verbal 
comments  were  recorded  during  the  second  and  third  simulations.  Many  more  comments  v/ere 
received  verbally  than  written. 

Pilot  ratings  were  completed  only  for  appropriate  maneuve.rs.  The  Cooper-Harper^  and 
PIO^  rating  scales  were  used  to  evaluate  configurations  during  flying  qualities  tasks.  The 
Cooper-Haiper  Rating  scale  is  shown  in  Figure  28  and  tlie  PIO  rating  scale  is  shown  in  Figure 
29.  Ratings  were  only  completed  by  pilots  who  had  been  trained  in  the  use  of  flying  qualities 
scales.  The  Cooper -Harper  and  PIO  ratings  were  completed  using  a  multi-function  display  in 
the  cockpit.  This  display  includes  an  electronic  implemeniarion  of  Cooper-Harper  Rating 
decision  tree  that  is  used  to  emphasize  the  decision  tree  process  and  descriptive  words  rather 
than  actual  numerical  latings.^^  The  value  of  pilot  commt  nts  to  support  and  describe  the  pilot 
rating  was  stressed  during  the  simulations.  Additionally,  a  "long-look”  evaluation  technique 
was  used  to  allow  the  pilot  adequate  time  and  plenty  of  evaluations  prior  to  completing  the 

rating.28.29 


37 


ADEQUACY  FOR  SELECTED  TASK  OR 
REQUIRED  OPERATION* 


AIRCRAFT 


DEMANDS  ON  THE  PILOT  IN 


PILOT 


CHARACTERISTICS  |  SELECTED  TASK  OR  OPERATION*  jRATING 


Excellent  !  Pilot  compensation  not  a  factor  for 

Highly  Desirable  i  desired  pertorrnance 


I  Good  I  Pilot  compensation  not  a  tacior  for 

Negligible  Deficiencies  '.desired  portormance. 


Fail,  Some  Mildly  Un-  .Minimal  pilot  co.tipansation  required 
pleasant  Deficiencies  I  tor  desired  performance 


'Very  Objectionable  but,  Adequate  performance  requires 
Tolerable  Deficiencies  .extensive  pilot  compensation 


Adequate  performance  not  attainable 


Minor  but  Annoying 
Deficiencios 

- j. 

Desired  performance  requires  4 

moderate  pilot  compensation 

Moderately  Objection¬ 
able  Deficiencies 

Adequate  performance  requires  i  ^ 

considerable  pilot  compensation  1 

■ 

1 

1 

Major  Deficiencies 

with  maximum  tolerable  pilot  compensa 
lion.  Controllability  not  in  question. 

7 

] _ 

1 

Major  Deliciencics 

jconsidorable  pilot  compensation  is 
!  required  lor  control. 

1 

J 

Major  Deficiencios 

1  Intense  pilcf  compensation  is  required 
to  retain  control. 

1 

9  ! 

I  { Major  Deficiencies 


iControl  will  bo  lost  during  some  portion  | 
of  required  operation 


‘Definition  of  required  operation  involves  designation  of  flight 
phase  and/or  subphases  with  accoiTipanying  conditions. 

Figure  28.  Cooper-Harper  Rating  Scale 


A  new,  NASA/Navy  developed  Pitch  Recovery  Rating  (PRR)  scale  was  also  used  during 
this  testing. This  scale  is  shown  in  Figure  30  and  is  structured  similar  to  the  Cooper- 
Harper  rating  scale  but  is  specialized  for  the  evaluation  of  nose-down  pitch  authority.  This 
scale  was  only  used  for  the  1-g  stabilized  pushover  maneuver  that  was  developed  in  Reference 
30.  Also,  this  scale  was  used  by  the  single  participating  pilot  who  was  trained  to  use  it. 

Time  histoiy  data  was  stored  to  magnetic  tapes  during  the  simulation.  Ail  of  the  basic 
aircraft  states,  pilot  inputs,  and  important  internal  simulation  code  parameters  were  saved  for 
future  use.  All  of  the  runs  were  saved  to  tape,  but  only  the  '  data"  runs,  as  described  above, 
were  later  retrieved  for  post-simulation  data  analysis.  Additionally,  audio  and  video  recording 
were  conducted  during  all  simulations.  The  video  tape  was  used  to  record  the  HUD  image,  the 
target  (when  in  the  HUD  field-of-view),  and  a  display  of  the  pilot  inputs. 
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Description 

Numerical  I 
Rating 

No  tendency  for  pilot  to  induce  undesirable  motions. 

1 

Undesirable  motions  tend  to  occur  when  pilot  initiates  abnjpt 
maneuvers  or  attempts  tight  control.  These  motions  can  be 
prevented  or  eliminated  by  pilot  technique. 

2 

Undesirable  motions  easily  inducea  when  pilot  initiates  abrupt 
maneuvers  or  attempts  tight  control.  These  motions  can  be 
prevented  or  eliminated  but  only  at  sacrifice  to  task 
performance  or  through  considerable  pilot  attention  and  effort, 

3 

Oscillations  tend  to  develop  when  pilot  initiates  abrupt 
maneuvers  or  attempts  tight  control.  Pilot  must  reduce  gain  or 
abandor^  task  to  recover. 

4 

Divergent  oscillations  tend  to  develop  when  pilot  initiates 
abrupt  maneuvers  or  attempts  tight  control.  Pilot  must  open 
loop  by  releas.ng  or  freezing  the  stick. 

5 

Disturbance  or  normal  pilot  control  may  cause  divergent 
oscillations.  Pilot  must  open  control  loop  by  releasing  or 
freezing  the  stick. 

6 

Figure  29.  Pilot  Induced  Oscillation  Rating  Scale 
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Control  Margin 
Adequacy 


Pitch  Raaponaa 
Characteristics 


Mission  Suitability/  Pilot 
Safety  Rating 


Figure  30.  Pitch  Recovery  Rating  Scale 
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Chapter  4 

Evolution  of  Maneuver  Development  Process  and  Maneuvers 

A  maneuver  development  process  and  new  evaluation  maneuvers  were  defined  during  this 
research.  The  fmal  recommended  maneuver  development  process  is  described  in  Chapter  2 
and  the  maneuvers  are  described  in  Chapter  5  and  Reference  3.  This  chapter  provides 
additional  information  that  describes  bow  the  maneuver  development  process  was  used  and 
refined  while  developing  new  evaluation  maneuvers.  The  information  in  this  chapter  will  be 
presented  in  an  order  that  parallels  the  steps  in  the  maneuver  development  process  and  follows 
the  work  conducted  in  a  chronological  order. 

Candidate  Maneuver  Definition  and  Screening 


The  first  step  of  the  maneuver  development  process.  Candidate  Maneuver  Definition  and 
Screening,  was  conducted  during  Phase  I  of  this  contract.  Phase  I  was  used  to  generate  a  wide 
range  of  potential  evaluation  maneuvers  and  then  sort  out  the  most  promising  maneuvers  for 
later  refinement  and  testing  during  Phase  II.  An  oveiview  of  tlie  Phase  I  process  is  shown  in 
Figure  31. 


Maneuvers 
Initiated  by  Pilots 


Maneuvers 
Initiated  by 
Engineers 


Figure  31.  Phase  I  Generated  a  Large  Datebaae  of  Potential  Maneuvers 
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A  "brainsiorming"  approach  was  used  to  develop  a  wide  selection  of  potential  maneuvers. 
During  this  step,  pilots  and  engineers  independently  identified  potential  maneuvers  based  on 
their  experiences,  'fhey  submitted  their  suggested  maneuvers  on  a  maneuver  description  fom 
like  that  shown  in  Figure  32.  The  pilots  and  engineers  were  asked  to  complete  Oitly  the 
sections  of  the  maneuver  description  form  that  applied  to  their  backgrounds.  The  form  was 
designed  to  have  pilots  suggest  maneuvers  based  on  operational  tasks  and  concentrate  on  the 
right-hand  sections  f  tlie  form.  As  a  result,  pilots  identified  operational  scenarios  tl  ..t  they 
had  experienced  or  observed  during  missions  or  training  exercises.  They  also  suggested 
operationally  meaningful  measures  of  merit  anti  performance  attributes  that  could  be  associated 
with  each  potential  maneuver.  Tlie  engineers  identified  maneuvers  that  they  thought  would 
isolate  design  parameters  and  be  useful  for  design  guidance.  The  engineers  then  continued  to 
complete  the  form  froi.i  left  to  right  as  their  backgrounds  allowed.  Additionally  a  literature 
search  was  conducted  to  find  existing  documented  maneuvers.  The  literature  search  revealed 
several  valuable  maneuvers  but  also  substantiated  the  need  for  a  uniform  methrod  of 
documenting  and  describing  evaluation  maneuvers. 
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Figure  32.  Original  Version  of  the  Maneuver  Description  Sheet 


This  first  step  in  the  Candidate  Maneuver  Definition  and  Screening  phase  resulted  in  294 
candidate  maneuvers.  They  ranged  widely  in  complexity  from  very  simple  to  veiy  complex 
maneuvers,  and  sevei-al  aircraft  classes  and  flight  categories  were  represented.  The  maneuver 
descriptions  were  not  complete  because,  as  described  above,  the  Review  Team  worked 
independently  and  only  completed  the  sections  of  the  maneuver  description  form  that  their 
background  allowed.  As  a  result,  the  maneuvers  submitted  by  pilots  tended  to  have 
information  in  the  right-hand  colunuis  while  the  engineers'  maneuvers  tended  to  be  complete 
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on  the  left  hand  side  of  the  form  shown  in  Figure  32.  At  this  point,  the  maneuvers  were 
studied  and  combined  because  several  very  similar  maneuvers  were  submitted.  In  some  ca.ses, 
the  pilots'  inputs  were  merged  with  the  engineers'  inputs  to  form  a  more  complete  maneuver 
description.  This  compilation  resulted  in  approximately  200  unique  potential  maneuvers. 

Of  these  200  maneuvers,  a  few  basic  maneuver  components  recurred  repeatedly.  For 
example,  simple  maneuver  segments  such  as  roll  and  capture,  pitch  and  capture,  level  turn,  or 
axial  acceleration  maneuvers  could  be  broken  out  of  larger  more  complex  maneuvers. 

However,  some  complex  maneuvers  could  not  be  broken  down  inio  sn\aller  maneuver 
elements.  These  observations  led  to  classification  of  maneuvers  into  one  of  the  tloee  following 
categories:  individual  maneuvers,  maneuver  sequences,  and  freestyle  maneuvers  as  shown  in 
Figure  33.  Individual  maneuvers  w'ere  defined  to  be  the  most  basic  element  of  a  maneuver  and 
could  not  be  broken  down  further.  Examples  of  individual  maneuvers  include  the  following: 
full  stick  pitch  pull,  nose-high  pushover,  and  a  36C*  roll  with  no  capture  required.  Maneuver 
sequences  can  be  visualized  as  combinations  of  individual  maneuvers.  A  pop-up  ground  attack 
maneuver  can  be  thought  of  as  a  maneuver  sequence  because  the  pilot  pulls  to  a  desired  pitch 
attitude,  climbs  to  a  given  altitude,  rolls  inverted,  pulls  to  and  captures  a  target,  then  rolls  back 
to  wings  level  while  tracking  the  target.  And  as  the  name  implies,  lreest}'le  maneuvers  allow 
the  pilot  a  great  deal  of  freedom  to  fly  the  maneuver.  Basically  only  the  start  condition  and  end 
condition  are  specified  for  a  freestyle  maneuver.  The  pilot  has  the  freedom  to  maneuver  in  any 
method  to  transition  from  one  state  to  the  other  .state.  An  example  freestyle  maneuver  would  be 
a  minimum  time  180'  heading  change  where  the  pilot  was  allowed  to  try  a  variety  of  tactics.  It 
was  determined  later  in  this  research  that  not  all  maneuvers  could  be  rigidly  categorized  as  a 
certain  type  of  maneuver,  but  these  definitions  arc  helpful  to  loosely  describe  the  maneuvers. 


Individual  Maneuvers  Maneuver  Sequences  Freestyle  Maneuvers 

Figure  33.  Maneuvers  Can  Generally  Ss  Classified  Into  Three  Categories 
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The  next  step  in  Phase  I  consisted  of  an  independent  review  of  all  of  the  maneuvers  by  the 
Review  Team.  This  is  the  first  time  that  Review  Team  members  were  able  to  review  candidate 
maneuvers  recommended  by  other  team  members.  Many  of  the  maneuvers  had  been  merged 
together  and  the  maneuver  description  forms  were  more  complete;  however*  some  blank 
columns  still  remained.  The  Review  Team  members  were  asked  to  comment  on  the  maneuvers 
and  fill  in  any  remaining  blanks  that  they  could.  During  this  process,  engineers  were  able  to 
complete  maneuvers  initiated  by  pilots  and  vice  versa.  Review  Term  members  also 
recommended  the  elimination  of  many  maneuvers.  Several  reasons  were  used  to  eliminate 
ir*ancuvers  including  the  following:  commonly  accepted  maneuvers  were  removed  because  of 
tlie  desire  to  augment  existing  evaluation  maneuvers,  maneuvers  that  were  not  dominated  by 
flying  qualities  or  agility  atttibutes  were  deleted,  ntaneuvers  that  only  isolated  deficiencies 
already  txpiosed  by  other  maneuvers  were  eliminated,  maneuvers  that  appeared  to  be  overly 
complex  to  set  up  and  fly,  and  maneuvers  that  were  not  demanding  enough  to  uncover 
deficiencies  were  dropped  from  consideration.  This  screening  process  reduced  the  number  of 
potential  maneuvers  to  89.  At  this  pwint,  the  Review  Team  independently  voted  for  the 
maneuvers  they  thought  were  the  most  important  to  continue  developing.  The  results  of  this 
voting  were  used  to  select  the  top  32  candidates  for  future  consideration. 

After  selecting  the  top  32  potential  maneuvers,  an  on-site  Review  Team  meeting  was  held  to 
discuss  and  further  develop  these  candidates.  This  step  represented  the  first  time  that  a  joint 
Review  Team  analysis  of  the  maneuvers  was  conducted.  All  of  the  32  maneuvers  were 
reviewed  a'‘.d  the  two-day  meeting  resulted  in  the  refinement  of  17  maneuvers,  the  elimination 
of  15,  and  tlie  identification  of  8  new  potential  maneuv  ers.  Figure  34  lists  each  group  of 
maneuvers,  reasons  for  eliminating  maneuvers,  and  documents  which  maneuvers  were 
successfully  developed  into  evaluation  maneuvers.  As  a  result.  Phase  1  was  completed  with  a 
list  of  25  potential  maneuvers  to  be  considered  for  flight  simulation  and  58  additional 
maneuvers  that  had  not  been  developed  as  fully.  Some  of  the  maneuvers  not  tested  in 
simulation  may  be  valuable  for  futiu-e  development;  therefore,  the  maneuvers  shown  in  Figure 
34  and  other  maneuver  candidates  are  contained  in  Appendix  A.  Some  of  the  maneuvers  in 
Appendix  A  may  be  redundant  v/ith  other  STEMS  maneuvers  or  may  not  prove  useful; 
however,  they  arc  provided  as  a  source  of  ideas  for  future  maneuver  development. 

Additionally,  maneuver  evaluation  fonns  and  simulation  comment  cards  were  developed  during 
Pha.se  I.  ITicy  were  later  refined  during  the  Phase  II  simulations,  and  the  final  versions  of 
these  forms  me  included  in  Chapter  2. 
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Maneuver  i  Refined 


Maneuver 

Notes 

295 

Stralglit  and  Leval  Acceloratiort 

Promising  maneuver,  rot  yet  developed  and  tes'eci  in  simulation 

302 

Straight  and  Lovol  Daceleration 

Promising  maneuve'.  not  yet  devebpsd  and  tested 

303 

T urning  Deceleration 

Promising  maneuver,  not  yet  developed  and  tested 

313 

PKch  AP.Itude  Capture 

Developed  in*c  STEM  7  -  ls‘ose-Up  Pitch  Angle  Capture 

312 

Maximum  Pitch  Pull 

Developed  Into  STEM  6  -  Maximum  Pitch  Pull 

316 

Pitch  Unload 

Use  existing  maneuver.  STEM  16  -  1-g  Ctabiiizad  Pusnover 

237 

Bar)k-lo-6ank  Roll 

Promising  maneuver,  not  yet  developed  and  tested 

307 

Loaded  Roll  and  Capture 

Developed  into  STEM  13  -  High  AOA  Roll  and  Capture 

310 

Lateral  Gross  Acquisition 

Use  existing  .-nanauver,  STEM  3  -  High  v\OA  Lateral  Gross  .Acq. 

286 

Maximum  Sideslip 

Promising  maneuver,  not  yet  devolopterl  and  tested 

79 

Arrest  High  Sink  Rato 

Promising  manouver,  vet  developeo  and  tested 

299 

Offset  Approach  to  Landing 

r^romising  maneuver,  not  yet  developed  ano  rested 

.305 

T urn  Entry 

Dropped  after  sim  and  TPS  last  -  Insensitive  to  Design  Parameters 

290 

Maximum  Rate  Level  Turn 

Developed  into  STEM  15  -  Mirimum  Time  1P0‘  Heaoing  Change 

14 

Acceleration  to  Loop 

Developed  mto  STEM  14  -  Minimum  Speed  Full  S.icK  Loop 

99 

Hampr.ar  Huad 

Promising  maneuver,  not  yet  developed  and  tested 

184 

Terrain  Folbwirig/Teriair.  Avoidance 

WL  pursuing  development  of  SlalortVDolohin 

f 

iXaneuvers  Etimiiiaied 

Maneuver 

Notes 

9 

Freestyifci  High-Speed  Acceleration 

Better  data  from  separate  p,;ci>  unload  anc  acceleraiicn  tasks 

192 

Maximum  Pitch  Pull  from  Level  "^uin 

Eliminate  in  favor  of  312 

250 

Pitch  Agility  Task 

Too  man/  maneuver  elomcr.is  to  analyze,  rot  c.^smtional 

306 

Maximum  Rate  Roll 

Learn  notliing  new  over  307 

15C 

LoedPd  Roll 

Learn  nothing  new  over  307 

249 

Roll  Agility  Task 

Similar  attributes  as  307  and  310  (307  and  310  more  r3|.satable) 

43 

Inctrumwm  Approach,  Pinal  Segment 

Not  demanding  eiiough 

306 

Heading  C  ipture 

Nrt  opoiational.  learn  nothing  new  over  othei  maneuvers 

58 

Rolling  Pull-Up 

Simliai  attributes  as  305,  pursue  305  instead 

267 

Dofansivq  Gun  Attack 

Complicated,  cxpeci  chaotic  lesults 

113 

Freestyle  High-Speed  Reversal 

Complicated,  expect  chaotic  results 

141 

Tran'itior;  frori.  Pop  to  Target  Attack 

Learn  nothing  ne  • 

98 

Tivo-Circle  uufueriy  Ve.llcal  Reverse 

Learn  nothing  new  ov  nr  ?07,  31  b 

247 

linloaded  Turn  Reversal 

Laarn  nothing  new  over  307,  310 

200 

Unnamed 

No  nev/  attributes  evaluated 

New  Maneuvers  Identified  j 

1  Maneuve; 

Notes 

313 

Go-Arnund  Maneuver 

Evaluate  transients  to  clran-up  aircraf'  and  >cx;elerate  ! 

320 

Optimum  Acceleration 

Freestyle  maneuver 

32.^ 

Unload  from  Clmax 

Evaluate  nose-down  control  1.  am  k.aded  condition 

324 

Sideslip  Tracking 

Directional  flying  qualities  task 

325 

Axial  Tiacking 

Axial  prertision  flying  qua'.tiss  task 

326 

Axial  Acc|uisitio:i 

Axial  decei  and  accel  acquisition  flying  nualities  task 

Note;  Nu'obtrs  shown  in  the  left  har^'i  column  are  temporary  STEM  numtxjrs  and  should  not  be 
curifusod  with  the  final  STEM  numbers  shown  in  the  notes  column  and  in  Reterence  3.  The 
numbers  in  the  left-hanc'  coiunin  can  be  used  to  locate  maneuver  descriptions  in  Appendix  A. 


Figure  34.  of  the  Review  Meetirig  Prior  to  Sitn':!etlon  Eftori 
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Maneuver  Development  and  ReHnement  Using  Piloted  Simulation 


Phase  II  of  this  contract  was  used  to  develop,  test,  and  refine  \.  va!uation  maneuvers  through 
piloted  simulations.  Three  simulations  were  conducted  and  each  simulation  was  structured  to 
have  a  slightiy  different  approach  and  objectives.  The  first  simulation  was  designed  to  be 
exploratory.  Several  maneuvers  were  defined,  preliminary  data  was  taken,  and  the  analysis 
tools  and  techniques  were  tested  and  evaluated.  A  period  of  data  analysis  was  conducted  after 
tlie  simulation  to  evaluate  the  maneuvers.  If  the  maneuvers  successfully  passed  the  Review 
Team  analysis,  then  the  maneuver  was  finalized.  Otherwise,  it  was  either  refined  in  the  next 
simulation  or  removed  from  future  consideration.  Also  adjustments  in  the  test  procedures  and 
analysis  techniques  were  matle  prior  to  the  second  simulation.  The  second  f  imulation  was 
used  to  develop  a  few  new  maneuvers,  but  it  mainly  served  as  the  primaiy  data  gathering 
effon.  Again,  tlie  maneuvers  were  reviewed  to  determine  which  required  additional  testing  in 
the  third  simulation.  The  final  simulation  was  then  used  to  complete  a  fmal  rcfuiement  of 
maneuvers,  develop  a  few  new  maneuvers,  validate  some  of  the  maneuvers  with  a  different 
simulation  model,  and  evaluate  the  flight  test  plan. 

The  firs*  simulation  was  designed  to  be  exploratory  sc  that  a  maximum  number  of 
maneuvers  could  be  investigated  and  the  simulation  test  approach  could  be  eva’uated.  The 
simulation  was  conducted  during  a  five-day  period  and  included  approximately  30  hours  of 
simulation  test  time.  Approximately  15  maneuvers  were  developed  and  additional  variations, 
such  as  testing  different  flight  conditions  and  capturing  various  pitch  attimdes,  were  flown  for 
several  of  the  maneuvcis  using  the  GENAIR  fighter  model  described  in  Chapter  3.  Design 
parameter  vaiiations  were  tested  for  each  maneuver  to  begin  investigating  the  sensitivity  of  the 
maneuver  to  primary  design  parameters.  A  limited  a  Kiuni  of  data  was  collected  daring  the 
‘irst  simulation  so  that  more  maneuvers  could  be  screened  and  developed.  Typically,  data  was 
gathered  from  a  single  pilot  for  each  maneuver.  (Data  was  taken  from  two  pilots  on  a  few 
maneuvers.)  A  total  of  six  pilots  particip?  ted  during  the  week  of  simulation  rnd  their  schedules 
were  such  that  there  were  always  four  pilots  participating  at  any  one  time.  There  were  always 
two  operational  pilots  and  two  test  pilots  available  to  fly  the  maneuver  and  help  in  the  maneuver 
development.  This  was  in'cnded  to  ensure  that  the  maneuvers  would  have  strong  links  to 
operational  and  test  requirements.  Additionally,  then?  were  four  engineers  tha»  participated  for 
at  least  some  portion  of  the  week. 

Several  observations  were  made  from  tiie  first  simulation  that  resulted  in  a  more  efficieni 
second  simulation.  First,  post- simulation  data  processing  indicated  that  data  was  required  from 
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at  least  wo  pilots  before  any  strong  conclusions  could  be  made  about  that  m  neuver.  Data 
from  a  minimum  of  two  pilots  is  needed  to  measure  the  pUot  variability  that  is  to  be  expected 
from  the  maneuver.  However,  data  fi'om  a  single  pilot  was  generally  sufficient  to  determine  if 
the  proper  design  parameters  and  ranges  were  being  tested.  Second,  it  was  determined  that 
only  one  operational  pilot  and  one  test  pilot  were  needed  at  any  one  time  to  support  the 
maneuver  development  and  data  gathering  process.  This  was  evident  because  the  two  test 
pilots  tended  to  agree  and  the  two  operational  pilots  tended  to  share  llie  same  perspective. 
However,  it  was  valuable  to  have  at  least  one  pilot  experienced  in  each  specialty  because  of  the 
unique  insights.  Another  conclusion  from  the  first  simulation  was  that  more  time  snould  be 
spent  flying  the  maneuveis  tind  less  time  should  be  spent  discussing  them.  Seeminglj  infinite 
discussions  could  be  resolved  much  quicker  by  trying  various  techniques  for  a  maneuver  in 
simulation.  Additionally,  some  of  the  simulation  tools  and  evaluation  forms  were  updated  as  a 
result  of  observations  during  the  simulation. 


The  second  simulation  was  designed  to  gather  the  bulk  of  the  data  for  this  contract  as  well 
as  begin  validating  the  maneuvers  with  a  more  complex  simulation  model.  This  simulation  was 
conducted  over  a  two-week  span,  that  included  7  days  of  testing  and  approximately  49  hours  of 


siniulation  test  time.  A, 
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ers  was  flown.  Tour  new  maiicuvcrs,  that  were  not 


developed  in  the  first  simulation,  were  developed  arid  tested  during  the  second  simulation.  The 
majority  of  tlie  simulation  time  was  devoted  to  testing  design  paranieter  variations  because  most 
of  me  maneuvers  had  been  developed  previously.  Data  was  gathered  from  one  additional  pilot 
to  augment  the  first  simulation  data  for  some  maneuvers.  At  least  two  pilots'  data  was  taken  in 
cases  where  the  test  matrix  needed  revising  between  the  first  and  second  simulations.  In  a  few 
cases,  data  was  gailieied  from  four  pilots  so  that  pilot  variability  could  be  investigated  in  more 
detail.  Some  of  the  simulation  time  was  used  to  begin  validating  the  maneuvers  in  a  high 
fidelity  F/A-18C  aircraft  m.odel.  The  maneuvers  were  developed  and  design  parameter 
variations  were  tested  using  the  GENAIR  fighter  model.  As  a  result,  the  nrore  complex  F/A- 
1  bC  model  was  used  to  check  the  flyability  of  some  of  the  maneuvers.  A  total  of  four  pilots 
participated  in  the  second  simulation  (two  pilots  were  available  diaring  the  first  week  of  testing 
and  two  differen*  pilots  were  available  during  the  second  week).  Orie  test  pilot  and  one 
c  peraiional  pilot  were  available  for  maneuver  development  and  data  gathering  at  any  one  time. 


The  second  simulation  was  much  more  efficient  than  the  first  bex^ause  of  the  progress  made 
and  lesso.ns  learned  during  the  fust  simulation.  Many  of  the  maneuvers  had  abe  dy  been 
developed  duiing  tlie  first  simulation,  so  some  of  them  were  simply  flown  for  additional  data 
while  others  were  slightly  modified  based  on  die  Review  Team  inputs.  Also,  the  test  matrices 
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had  been  previously  evaluated  and  either  verified  or  refined  prior  to  the  second  simulation,  so 
the  data  gathering  tended  to  opei  ate  more  smoothly.  In  general,  the  simulation  time  was  used 
more  effectively  because  more  time  was  flying  maneuvers  and  trying  variations  and  less  time 
was  spent  discussing  them.  Tlie  people  participating  were  also  used  more  effectively  because 
only  two  pilots  and  rwo  engineers  were  on  hand  at  any  one  time.  Finally,  tlie  second 
simulation  spanned  two  weeks,  so  it  allowed  simple  modifications  to  the  test  matrices  or 
simulation  setup  between  weeks. 


The  third  simulation  was  used  to  gather  final  data,  test  if  this  evaluation  maneuver  process 
could  be  applied  to  transport  class  aircraft,  validate  the  maneuvers  with  a  high  fidelity 
simulation  model,  and  evaluate  the  flight  test  plan.  The  majority  of  the  simulation  was 
conducted  in  a  5  day  period  that  included  approximately  40  hours  of  simulation  test  time. 
Additional  days,  and  approximately  14  hours  of  simulation  test  time,  were  used  for  more 
validation  and  to  evaluate  the  flight  test  plan.  Founetn  maneuvers  were  flown  during  this 
simulation,  including  4  new  maneuvers.  The  data  gatheiing  procedures  for  the  third  simulation 
were  slighdy  modified  from  the  second  simulation.  During  the  second  simulation,  all  the 
maneuvers  were  flown  for  numerical  measure  of  merit  data,  and  subsequent  data  analysis 
concentrated  of  quantitative  data.  However,  during  the  third  sinvulution,  some  marieuvers  were 
flown  only  to  collect  pilot  comments  because  previous  data  analyses  had  indicated  the  inability 
to  use  quantitative  data  from  certain  maneuvers.  In  particular,  some  of  the  flying  qualities 
maneuvers  were  flown  solely  for  comments  and  not  for  measure  of  merit  data  because  similar 
maneuvers  resulted  in  very  poor  quantitative  data.  Also,  some  of  the  previously  developed 
maneuvers  were  tested  to  determine  if  they  could  be  flown  with  alternate  command  types,  such 
as  an  angle  of  attack  rate  command  system.  ITiese  maneuvers  were  flov/n  briefly  to  check  for 
flyability  and  gross  trends  with  design  parameter  variations. 


The  flight  test  plan  portion  of  the  third  simulation  was  designed  to  gather  much  different 
information  than  the  design  parameter  testing.  A  simplified  NASA  High  Alpha  Research 
Vehicle  (HARV)  model  was  used  .so  that  tlie  performance  characteristics  would  be 
representative  of  flight  test  fDic  HARV  is  currently  the  most  likely  aircraft  to  be  used  to 
validate  STEMS  in  flight  test.)  This  test  was  conducted  to  identify  any  changes  in  setup  that 
Tiiight  be  required  for  the  HARV  and  to  get  a  preliminary  idea  of  the  flight  tm>e  required  to 
validate  STEMS.  Additional  simulation  work  with  a  higher  fidelity  HARV  model  is 
recommended  prior  to  flight  test  Refeicncc  5  contains  tiie  flight  test  plan  and  additional  details 
of  this  testing. 
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Validation  of  most  of  the  maneuvers  using  a  high-fidelity  simulation  model  (MuSIC)  was 
also  conducted  duiing  the  third  simulation.  A  relatively  simple  model  (GENAIR  fighter  model) 
was  used  to  develop  the  maneuvers,  so  a  complex  simulation  model  with  different  performance 
and  flying  qualities  was  used  to  help  validate  the  maneuvers.  The  maneuvers  were  flown  to 
determine  if  the  setups  were  tolerant  to  wide  aircraft  performance  changes  and  if  the  maneuvers 
were  flyable  with  a  high  fidelity  simulation  model.  All  of  the  maneuvers  that  were  tested  with 
the  MttSIC  model  proved  to  be  flyable  and  applicable.  The  MuSIC  model  was  also  used  to 
evaluate  limited  design  variations  with  some  of  the  maneuvers. 

The  GENAIR  transport  iriodel  was  flown  during  the  third  simulation  to  test  the  ability  to  use 
this  maneuver  development  process  for  other  classes  of  aircraft.  Only  four  maneuvers  were 
tested,  but  the  results  indicated  that  this  process  could  be  extended  to  other  classes  of  aircraft 
The  'Tanker  Boom  Tracking"  maneuver  (STEM  18)  was  found  to  be  a  useful  evaluation 
maneuver,  the  "Tracking  in  Power  Approach"  maneuver  (STEM  19)  and  the  "Nose-Up  Pilch 
Angle  Capture"  maneuver  (STEM  7)  appeared  to  be  possibly  valuable  but  could  use  additional 
validation;  and  a  flight  path  capture  maneuver  was  tried  but  was  not  found  to  be  useful.  Design 
parameter  variations  were  also  tested  with  the  transport  aircraft,  and  the  data  obtained  indicates 
that  the  STEMS  process  can  be  used  in  the  design  of  a  transport  class  aircraft  also. 

Data  Analysis  and  Evaluation  of  Maneuvers 

Periods  of  data  analysis  were  performed  between  each  simulation  to  evaluate  and  refine  the 
maneuvers.  This  analysis  was  used  to  determine  the  following  chaiacteristics:  repeatability  and 
testability  of  the  maneuver,  ability  of  maneuver  to  provide  useful  design  information,  value  of 
comments  and  measure  of  merit  generated,  relation  cf  the  maneuver  to  operational  applications, 
and  the  potential  for  GLOC  or  spatial  disorientation.  This  analysis  and  review  was  used  to 
modify  maneuvers  that  were  deficient  in  some  way.  A  maneuver  was  accepted  as  a  STEM  if  it 
successfully  met  most  of  these  criteria.  A  maneuver  was  dropped  if  it  failed  many  of  these 
checks.  A  combination  of  quantitative  and  qualitative  analyses  were  conducted  on  the 
simulation  data  to  evaluate  each  maneuver.  Several  pieces  of  data  were  generated  from  the 
design  parameter  variation  testing,  including  the  following:  numerical  measures  of  merit ,  pilot 
comments,  nilot  ratings,  maneuver  summary  comment  cards,  and  human  factors  analyses.  All 
of  this  data  was  reviewed,  as  shown  in  Figure  35,  to  evaluate  the  relative  success  or  failure  of 
the  maneuvers.  Tlie  data  was  also  analyzed  for  its  applicability  to  the  design  process.  Each 
maneuver  was  then  either  accepted,  elimina.'ed,  or  modified  based  on  Review  Team  analysis  of 
the  simulation  data. 
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Maneuver 

Descriptions 


Measures 
of  Merit 


Pilot 

Comments 


Time  History 
Plots 


Human  Factors 
Analysis 


Review  Team 


•  Revise  Maneuver  Descriptions 

•  Identify  Promising  Maneuvers 

•  identify  Promising  Measures  of  Merit 

•  Suggest  New  Maneuvers 


FlQura  35.  Review  Team  Sources  of  Data  to  Evaluate  Each  Maneuver 


Summary  results  will  be  sliown  in  Chapter  5  for  each  maneuver  that  was  accepted  as  a 
STEM.  An  example  will  be  used  in  this  chapter  to  describe  the  steps  conducted  during  the  data 
analysis  and  maneuver  evaluation.  A  much  mcne  complete  set  of  data  fcK*  each  maneuver  is 
included  in  Reference  4.  The  summary  results  shown  in  this  report  simply  demonstrate  sample 
findings  and  sensitivities.  In  general,  the  results  for  each  maneuver  are  dependent  upon  the 
design  parameters  chosen  and  the  range  of  variation  tested.  For  example,  a  valuable  evaluation 
maneuver  might  be  sensitive  to  changes  in  short  period  frequency  but  not  very  sensitive  to  time 
delays.  This  maneuver  could  be  judged  mconectly  if  it  were  evaluated  by  cmly  testing  time 
delay  variations.  The  range  of  design  parameter  variatitm  conducted  also  strongly  influenced 
the  measure  of  merit  analysis.  Therefore,  tlie  results  shown  in  these  reports  arw  not  meant  to  be 
an  exclusive  list  of  design  parameters  for  each  maneuver. 


Quantitative  Data  Analysis 


The  quantitative  data  analysis  was  conducted  by  calculating  time  history  measures  of  merit 
and  evaluating  the  ability  to  use  this  infexmation  to  modify  a  design.  The  goal  of  this 
procedure  was  to  evaluate  the  sensitivity  of  tlte  measure  of  merit  to  design  parameter  variations 
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and  to  pilot  variability.  This  was  done  to  isolate  measures  of  merit  that  were  sensitive  tc»  design 
parameters  but  exhibited  little  pilot  variability.  Measures  of  merit  with  these  characteristics 
could  be  "trusted"  for  design  guidance.  In  summary  ,  this  process  compared  the  change  in  a 
measure  of  merit  due  to  a  design  parameter  variation  to  the  variation  that  was  due  to  differences 
in  pilots.  The  result  is  a  cross  matrix  that  identifies  how  well  each  measure  of  merit  can  be 
used  to  evaluate  the  changes  in  each  design  parameter  for  a  given  maneuver.  It  was  found  that 
some  measures  of  merit  might  effectively  measure  one  design  parameter  variation  but  not 
another.  Tnerefoie,  it  is  dangerous  to  rely  on  a  single  measure  of  merit  when  making  a  variety 
of  design  parameter  changes. 

A  screening  process  was  conducted  for  each  maneuver  to  select  the  most  appropriate 
measures  of  merit.  First,  the  Review  Team  generated  a  list  of  assented  potential  measures  of 
merit.  A  large  number  of  measures  of  merit  were  calculated  ftom  the  time  history  data  using  an 
automated  process.  The  measures  of  merit  were  mtentionally  selected  to  be  simple  to  calculate, 
and  therefore,  easier  to  measure  in  a  flight  test  environmenL  ITiey  were  also  selected  to  be 
meaningful  from  both  design  and  operational  standpoints.  Measures  of  merit  that  obviously 
were  not  applicable  to  a  certain  maneuver  were  not  calculated  for  that  maneuver,  but  any  that 
seemed  even  remotely  possible  were  investigated.  Figure  36  shows  a  list  of  the  measures  of 
merit  that  were  considered  during  this  study.  This  list  is  meant  to  be  repiesentative  of  typical 
measures  of  merit  and  is  not  intended  to  an  ail*encompassing  set  All  of  the  measures  of  merit 
were,  calculated  from  simple  time  histwy  signals  that  are  readil)'  available  from  a  simulation 
model,  and  most  should  be  available  from  flight  test.  The  measures  of  merit  are  described  in 
more  detail  in  Reference  4. 

Statistical  tests  were  conducted  after  calculating  the  measures  of  merit  to  dcteiminc  which 
measures  were  sensitive  to  design  parameter  variations,  'fhe  statistical  calculations  were  used 
to  isolate  the  effects  of  each  design  parameter  variation  arid  the  amount  of  pilot  variability 
present  in  each  measure  of  merit.  The  results  of  the  statistical  analyses  were  summarized  m 
both  numerical  and  graphical  forms  for  the  subsequent  Review  Team  analysis.  Several 
analyses  were  conducted  for  certain  maneuvers.  For  example,  several  different  pitch  attitudes 
were  used  to  test  a  pitch  angle  capture  maneuver.  A  separate  statistical  analysis  was  conducted 
for  each  angle  capture.  Also,  multiple  analyses  were  conducted  to  compare  the  results  from 
fractional  factorial  testing  to  full  factorial  testing  when  enough  data  was  available.  A  complete 
set  of  data,  including  the  numerical  and  graphical  summaries,  is  included  in  Reference  4,  and 
some  sample  sunutiary  information  is  shown  in  Chapter  5. 
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Figure  36. 


Description 

Time  to  pitch  through  an  X'  pitch  attlbjde  change 

Maximum  lift  coefficient  attained  during  the  maneuver 

Time  at  which  CLMAX  was  attained 

Average  pitch  acceleration  over  the  first  X  seconds 

Pitch  acceleration  X  seconds  after  inidation  of  the  maneuver 

Maximum  pitch  acceleration 

Time  at  which  maximum  pitch  acceleration  occurs 

Maximum  pitch  rate 

Time  at  which  maximum  pitch  rate  occurs 
Pitch  rate  at  X  seconds 
Maximum  AOA  rate 

Time  at  which  maximum  AOA  rate  occurs 
Angle  of  attack  rate  at  X  seconds 
Maximum  load  factor 

Tme  at  which  maximum  load  factor  occurs 

Maximum  load  factor  rate 

Time  at  which  maximum  load  factor  rate  occurs 

Maximum  incremental  pitch  attitude 

Time  at  which  maximum  pitch  attitude  occurs 

Maximum  angle  of  attack  attained  during  maneuver 

Time  at  which  maximum  AOA  occurs 

Angle  of  attack  at  X  seconds 

Change  in  angle  of  attack  from  initial  time  to  final  time 

Time  to  reach  SO*  angle  of  attack 

Time  from  initial  time  until  capture  occurs 

Time  to  complete  the  maneuver 

Settle  time  (time  to  capture  after  the  target  first  enters  the  error  band) 

Increment  in  altitude  between  initial  time  and  finai  time 

Increment  in  heading  between  initial  time  and  rinal  time 

Time  to  achieve  the  increment  in  heading  change 

Wind  axis  bank  angle  at  the  capture 

Maximum  stability  axis  rcil  rate  attained 

Time  at  which  maximum  stability  axis  roll  rate  occurs 

Maximum  stability  axis  roll  acceleration  attained 

Time  at  which  maximum  stability  axis  roil  acceleration  occurs 

Maxiirium  roll  deceleration  occurring  from  a  lateral  stick  cross-check 

Bank  cvershool  (integral  of  stability  axis  roll  rate  from  cross-check  until  zero  rate) 

Specific  excess  power  at  the  final  time 

Increment  in  specific  energy  between  the  initial  time  and  the  final  time 
Maximum  rate  of  change  of  equivalent  airspeed 
Increment  in  oquivaient  airspeed  between  the  initial  time  and  the  final  time 
Maximum  flight  path  rate 

Time  at  which  the  maximum  flight  path  rate  occurs 

Root  Mean  Square  of  longitudinal  stick  position 

Root  Mean  Square  of  lateral  stick  position 

Root  Mean  Square  of  elevation  tracking  error 

Root  Mean  Square  of  azimuth  tracking  error 

Longitudinal  position  deviation  upon  touchdown  for  the  landing  task 

Lateral  position  deviation  upon  touchdown  for  the  landing  task 

Touchdown  speed  deviation  for  the  landing  task 

Cooper-Harper  Rating 

Pitch  Recovery  Rating 

Roll  rate  at  X  seconds 

Roll  acceleration  at  X  seconds 

Root  Mean  Square  of  flight  path  error 

Mwituret  of  Ktorll  Calcolaitod  During  th«  Gontric  Fightar  Tostlng 
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An  example  maneuver  and  its  corresponding  data  analysis  will  be  used  to  help  describe  and 
document  the  maneuver  evaluation  process  used  few  this  contract.  A  low-speed,  maximum 
pitch  maneuver  will  be  used  as  an  example.  This  is  a  very  simple,  qjen-loop  maneuver 
designed  to  test  the  ma.\iraum  pitch  capabilities  of  an  airciafL  The  pilot  first  stabilizes  at  the 
desired  AOA  and  airspeed;  then  he  perfmns  an  aggressive  fiill-aft  stick  input  and  continues  to 
hold  aft  stick.  The  maneuver  is  terminated  when  the  aircraft  reaches  its  maximum  pitch 
attitude.  Variations  in  short  period  frequency  (WSP),  shewt  period  damping  (ZSP),  and 
maximum  attainable  angle  of  attack  (AOAMAX)  were  tested  fw  this  example. 

A  graphical  summary  of  data  from  the  maximum  pitch  pull  analysis  is  shown  in  Figure  37. 
The  bars  indicate  the  average  value  for  each  measure  of  merit  for  each  level  of  the  design 
paiameter  tested.  The  first  six  bars  on  each  graph  indicate  the  average  measure  of  merit  value 
for  each  variation  in  design  parameter.  The  difference  between  the  light  and  dark  bars  indicates 
the  change  in  measure  of  merit  that  is  due  to  the  change  in  that  design  parameter.  The  last  two 
bars  on  each  graph  indicate  the  overall  average  me^ure  of  merit  value  for  each  of  the  two 
pilots.  The  change  between  these  last  two  bars  indicates  the  difference  that  can  be  attributed  to 
pilot  technique.  A  statistical  significance  level  is  also  associated  with  each  of  these  sets  of  bars 
to  indicate  the  credibiiiiy  ot  the  statistical  calculations  (amount  of  noise  «■  unaccounted  for 
factors),  llie  significance  level  is  not  shown  in  Figure  37,  but  it  was  taken  into  consideration 
during  the  evaluation  process.^ 

The  sensitivity  of  each  measure  of  merit  to  each  design  parameter  and  to  pilot  variability  was 
analyzed  and  grouped  into  categories  to  indicate  the  relative  strength  of  that  sensitivity.  The 
percent  change  in  a  measure  of  merit  due  to  a  design  parameter  change  was  combined  with  its 
statistical  significance  to  Indicate  if  there  existed  a  strong,  potentially  strong,  potentially  weak, 
or  a  weak  degree  of  sensitivity.  If  the  dark  and  light  bars  in  Figure  37  vary  considerably  and 
the  statistical  tests  indicated  a  high  degn  e  of  confidence  in  the  average'  '  it  is  considered  a 
strong  sensitivity.  As  the  relative  change  decreased  in  magnitude  or  the  statistical  sigrificance 
declined,  then  tlie  sensitivity  was  classified  at  a  lower  level.  A  summary  of  this  analysis,  for 
the  maximum  pitch  pull  maneuver,  is  shown  on  the  left  hand  side  of  Figure  38.  'fhe  pilot 
variability  for  a  n'reasurc  of  merit  was  judged  by  comparing  the  difference  in  averages  due  to  a 
design  parameter  variation  to  the  difference  in  averages  between  the  pilots.  If  the  change  due  to 
tlie  design  parameter  was  much  larger  than  Uie  change  due  to  pilots,  that  combination  of  design 
parameter  and  measure  of  merit  was  considered  to  have  minimal  pilot  variability.  IncreasLng 
amounts  of  pilot  variability  were  labeled  as  "some  variability"  or  "large  variability."  A 
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Value  Oblaiiisd  With  Design  Parameter  at  (  )  Level 
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Figure  37.  Example  Measure  of  Merh  Data  foi  Maximum  Pitch  Puli  Maneuver 

'fhe  sensitivity  to  design  parameters  was  then  combined  with  the  sensitivity  to  pilot 
variability  to  form  an  overall  indication  of  the  ability  to  use  a  particular  measure  of  merit  when 
making  modifications  to  a  design  parameter.  Figure  39  shows  a  summary  of  these  overall 
sensitivities  for  tlie  example  pitch  pull  maneuver.  The  dark  regions  indicate  combinations  of 
design  parameters  and  measures  of  merit  that  can  be  trusted  during  the  evaluation  of  an  aircraft. 
The  dark  gray  combinations  represent  a  slightly  reduced  confidence  for  design  applications  ajid 
the  light  gray  or  white  regions  arc  probably  unacceptable  for  designers.  A  detailed  description 
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Minimai  Pilot  Variability 
Some  Pilot  Variability 
Large  Pilot  Variability 


Sensitivity  to  Design  Parameters 
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Sensitivity  to  Pilot  Variability 
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Time  to  Pitch  Through  15  deg 

Avti  Initial  Pitch  Accel  Over  0.25  sec 

Pitch  Acceleration  at  0.25  sec 

Max  Pitch  Acceleration 

Time  of  Max  Pitch  Acceleration 

Max  Pitch  Rate 

Time  of  Max  Pitch  Rate 

Pitch  Rate  at  1 .0  sec 

Max  Angle  of  Attack  Rate 

Time  of  Max  AOA  Rate 

Angle  of  Attack  Rate  at  1.0  sec 

Max  IncreiTiental  Pitch  Attitude 

Time  of  Max  Pitch  Attitude 

maxirnuiH  Arigle  cf  AUauk 

Time  of  Max  Angle  of  Attack 

Angle  of  Attack  at  3.0  sec 

Max  Acceleration/Oeceleration 


FIgura  38.  Design  Parameter  Variations  anil  Pilot  Variability 


of  tlic  rules  used  to  classif)'  the  sensitivities  and  pUot  variability  is  included  in  Reference  4. 
Typical  overall  sensitivities  will  be  shown  for  several  maneuvers  in  Chapter  5.  Measures  of 
merit  or  design  parameters  that  fell  into  the  lower  two  categories  will  be  removed  from  the 
figures  in  Chapter  5  for  simplicity.  A  complete  set  of  sensitivity  figures  is  included  in 
Reference  4. 


It  became  obvious  after  the  second  simulation  that  some  of  the  maneuvers  did  not  tend  to 
generate  good  measures  of  merit  In  general,  maneuvers  that  were  highly  closed-loop,  such  as 
tracking  tasks,  failed  to  produce  many  successful  measures  of  merit.  Therefao.  the  measure 
of  merit  analysis  was  not  performed  cn  similar  maneuvers  that  were  flown  during  tl»e  third 
simulation.  This  analysis  also  was  not  conducted  on  tests  that  were  used  to  validate  the 
maneuvers  with  the  MuSIC  simulation  model  cff  tests  used  to  validate  the  maneuvers  for 
alternate  pitch  command  systems. 
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Figure  39.  Exampio  Ovarall  Sansithrlilaa  o!  Maaaurea  of  Martt  for  Max  Pitch  Pull 


Qualitative  Data  An^'tysis 


Qualitative  data  was  also  studied  fee  each  potential  evaluaticHt  maneuver.  This  data  includrxl 
the  following-  pilot  comments,  pilot  ratings,  and  simulation  comnicnt  eaids.  The  Review  Team 
was  primaiily  looking  for  key  comments  and  data  that  can  be  used  to  direct  design 
modifications.  They  compared  pilot  opinions  to  see  if  cwisisterit  chanvctcristics  were  observed 
and  if  ctnistructive  comments  that  isolated  aircraft  attributes  were  obtained.  Pilot  ratings  were 
also  examined  to  detennine  if  the  ratings  were  consistent  with  the  comments  received  and  if  the 
ratings  seemed  leasonable  for  the  dynamics  being  tested.  TIus  step  requued  tlw  experience  of 
flying  qualities  engineers  to  evaluate  the  quality  and  content  of  the  comments  re  larvc  to  the 
design  parameter  changes  tested.  Additicxial  valuable  infotmation  was  obtainou  from  the 
summary  comments  given  by  the  pilots  during  the  simulation.  The  pilots  dcsc.ri‘icd  attr.butes 
of  a  maneuver  and  their  pen.eption  of  it  immediately  after  evaluating  all  of  the  de  sign  ptirameier 
variadens  by  using  the  comment  card  shown  in  Chapter  2,  Figure  15.  All  of  this  infoimdoa 
was  used  to  understand  the  maneuver  better  and  suggest  possible  icfincmevtts  if  necessary. 
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T.,e  qualitativ*;  data  is  very  difficult  to  summarize,  so  the  conqilete  pilot  comments,  ratiu^,  and 
ies))onse.’>  to  the  simuladoa  comment  caid  are  included  m  Reference  4,  and  only  summary 
conclusions  about  the  ability  or  inabiliQr  of  a  maneuver  to  generate  useful  qualitative  Ja^a  vrill 
be  included  in  Chapter  5. 

Human  Factors  Analysis 

All  of  Uk*  simulations  were  conducted  in  a  fixed-base  dome,  so  additional  analyses  were 
peifoimed  to  idendj^'  potential  motion  effects.  The  linear  and  angular  accelerations  experienced 
by  a  pilot  in  flight  depend  upon  the  aircraft  dynanucs,  locatiori  of  the  pilot  stariovt,  cockpit 
configuration,  and  other  aira  aft  dependent  characteristics.  Addidcnal  factors  such  as  inner  ear 
and  brain  stem  anatomy,  diet,  sleep  patterns,  vision,  and  weather  also  play  a  role  in  motion 
effects  on  a  pilot.  However,  analyses  were  conducted  to  try  to  identify  maneuvers  thrt  would 
be  potendaliy  dangerous  in  flight  More  specifically,  GLOC  and  .spadal  disoientadon  were 
considered.  A  qualitadve  assessment  of  the  maneuvers  was  perfonned  before  the  fust 
simuladon  to  try  to  isolate  potendal  modon  consideradons.  Time  histexy  data  was  recorded  and 
pilot  quesdonnaiies  were  cor^ipleted  during  the  simuladons  to  determine  if  any  pioblems  co*ild 
be  andcipated  during  flight.  Load  factor  time  history  data  for  all  of  the  Tnancuvers  was 
evaluated  using  the  Dynamic  Acceleration  Canputc  Model  (DACM),  'ihe  DACM  predicted  no 
possibility  of  GLOC  in  the  maneuvers.  This  is  primarily  due  to  the  low  speed  nature  of  the 
maneuvers  and  the  subsequently  minimal  load  factws  experienced.  Unfortunately,  due  to  the 
extremely  complex  nature  of  spadal  disorientation,  it  is  difficult  to  predict.  Some  computer 
models  are  cunendy  being  developed  but  were  net  used  in  this  research.  Instead,  a  qualitadve 
evaluadcHi  was  performed  based  on  answers  to  the  hutnan  factors  pilot  questionnaire  shown  in 
Chapter  2,  Figure  16.  The  rating  scales  used  to  summarize  susccpiibility  to  GLOC  and,  spadal 
disorientation  arc  shown  in  Figure  40.  Figure  41  contains  the  results  of  the  hiunan  facia's 
analysis  of  the  STEMS  maneuvers  and  indicates  no  strong  potential  cases  of  GIXK!  a  spatial 
disorientation. 

Evaluation  of  Maneuvers 

A  Review  Team  analysis  of  the  simulation  data  was  conducted  after  the  fust  and  second 
simulations.  The  Review  Team  was  furnished  with  the  following  infooKwiticr,:  maneuver 
descriptions,  statistical  sunMuarics  of  the  measure  of  merit  analysis,  time  histoiy  plots,  piia 
comioents  and  ratings,  responses  frixn  the  simulation  comment  cards,  and  results  of  the  human 
factors  analysis.  ITiey  were  asked  to  ro'iew  the  pieces  of  data  that  were  most  meaningful  to 
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them.  Each  Review  Team  member  analyzed  the  data  from  a  slightly  different  pei'spective  and 
responded  with  different  types  of  inputs.  The  primary  goal  of  the  Review  Team  analysis  was 
to  evaluate  the  quality  of  the  data  generated  from  each  maneuver,  determine  the  applicaLiilit\  of 
each  maneuver  to  the  design  process,  and  judge  its  relevance  to  operational  use. 


Gz-Tolerance  * 

- - j 

Gx.y  -  Tolerance  *  | 

]  Rating 

Injury 

Description 

Rating 

Injury 

Description 

B 

None 

No  Pred.  Difficulties 

0 

None 

No  Pred.  Difficult'es 

1 

Possible 

Possible  gray/blackout 
>4-6  instantaneous 

1 

Possible 

Possible  Injury 
(0-1 G) 

2 

Probable 

Probable  gray/blackou* 
6-8  instantaneous 

2 

Probable 

Probable  injurv 
(1-2G; 

3 

Definite 

Definite  blackout 

7+  instantaneous 

3 

Definite 

Incapacitating  injury 
(2+G'b) 

*  Aircrew  wearing  standard  G  suit/straining 


I  Spatial  Disorientation  **  | 

Rating 

Category 

Description 

0 

None 

No  predicted  difficulties 

1 

Possible 

Possibility  of  SD  (2  planes  or  moveiment/rapid) 

2 

Probable 

High  Probability  of  SO  (2-3  plancs/rapid; 

3 

Definitely 

SD  Inevitable  (3  planes/rapid  movement) 

Assumption:  clear  VFR  day 

Figure  40.  G  Tolerance  and  Spatial  Disorientation  Rating  Scales 


Some  key  considerations  were  identified  during  the  Review  Teair.  analysis,  it  was 
observed  that  differences  in  the  data  due  to  design  parameter  variations  needed  to  be  niiic!’. 
greater  that  tlie  amount  of  pilot  variability  in  tiic  data.  This  is  an  important  consideration  for  all 
of  the  data,  but  was  most  ca.sily  applied  to  the  quantitative  measure  of  merit  rtnal)ses.  In  other 
words,  if  measures  of  merit  are  to  be  calculated  for  a  maneuver,  it  is  e'^sential  ihui  they  be 
relatively  insensitive  to  variations  between  pilots  yei  sensitive  to  variations  in  the  design.  It 
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was  concluded  that  measures  of  merit,  and  other  types  of  data,  were  useless  to  a  designer  if  the 
data  is  mote  sensitive  to  the  pilot  than  the  configuration  dynamics.  Two  major  factors  were 
used  to  substantiate  Utis  opinion.  First,  it  cannot  be  assumed  that  the  same  pilot  will  be 
available  during  all  of  design  and  flight  test,  so  data  that  is  to  be  used  in  a  comparadve  fashion 
needs  to  be  relatively  tree  of  pilot  variability.  Second,  the  data  should  have  a  strong  coirelarion 
back  to  design  parameters.  If  a  design  parameter  is  adjusted,  then  the  data  must  accurately 
reflect  any  performance  changes  rather  than  being  overwhelmed  by  pilot  variaoility.  Pilot 
comments  and  ratings  v'cre  also  considered  as  key  sources  of  evaluation  data.  Therefore,  they 
were  examined  for  useful  design  feedback  information  by  comparing  the  comments  to  the 
parameter  variations  tested  and  locking  for  key  comments  that  would  oe  meaningful  for  design 
modifica:  ons.  After  reviewing  the  simulation  data,  eacn  Review  Team  member  completed  the 
evaluation  form  shown  in  Chapter  2,  Figure  13.  Th’s  form  was  intended  to  provide  u.seful 
information  to  improve  a  maneuver.  The  Review  Team  members  were  also  asked  to  identify 
each  maneuver  as  being  sufficiently  complete,  needing  refinement,  or  not  worth  retaining. 


Maneuver 

Number  Name 

Human  Factors  Estimate 

Gz  Gx,y  SD 

DACM 

Results 

SD  Pilot 
Responses 

Tracking  -  High  AOA  Sweep 

U 

0 

. 

0 

No  GlOC 

- 

4 

Dual  Attack 

1 

0 

0 

No  GLOC 

0  Yes,  2  No 

5 

Rolling  Defense 

1 

0 

1 

No  GLOC 

2  Yes,  0  No 

6 

Maximum  Pilch  Pull 

1 

0 

0 

No  GLOC 

0  Yes,  2  No 

7 

Nose-Up  pitch  Angle  Capture 

1 

0 

1 

No  GLOC 

1  Yes,  2  No 

9 

Pitch  Rate  Reserve 

1 

0 

0 

No  GLOC 

■ 

11 

Sharkenhausen 

1 

0 

0 

No  GLOC 

0  Yes,  3  No 

12 

High  AOA  Roll  Reversal 

1 

0 

0 

No  GLOC 

1  Yes,  1  No 

16 

1  -g  Stabilized  Pushover 

0 

0 

0 

Nc  GLOC 

0  Yes,  2  No 

17 

J-Turn 

1 

0 

0 

No  GLOC 

- 

Offset  Approach  to  Landing 

1 

0 

1 

No  GLOC 

0  Yes,  2  No 

Data  not  analyzed  for  the  following  STEMS: 

2  High  AOA  Tracking  -  Maneuver  tested  under  other  research. 

3  High  AOA  Lateral  Gross  Acquisition  -  Maneuver  teste  d  under  other  researeh. 

8  Crossing  Target  Acquisition  and  Tracking  -  Benign  motion,  not  analyzed. 

1 0  High  AOA  Longitudinal  Gross  Acquisition  -  Maneuver  tested  under  other  research. 

13  High  AOA  Roll  and  Capture  -  Same  motion  environment  as  12,  not  analyzed 

14  Minimum  Speed  Tull  Stick  Loop  -  Not  analyzed  due  to  time  constraints. 

15  Minimum  Time  180'  Heading  Change  -  Net  analyzed  due  to  time  constraints. 

18  Tanker  Boom  Tracking  -  Benign  motion,  not  analyzed. 

19  Tracking  in  Power  Approach  -  Benign  motion,  not  analyzed. 


Figure  41.  Summary  of  Human  Factors  Pradlctiona  and  Analysis 


Flight  Test  Validation 


■Die  maneuvers  defined  under'  this  contract  were  develc^>cd  in  fixed-base  simulation; 
therefore,  it  is  necessary  to  validate  them  with  in-flight  testing.  This  validatioa  has  been  started 
by  the  US  Air  Force  Test  Pilot  School  as  two  class  projects.  Six  STEMS  maneuver  candidates 
("High  AOA  Tracking"  -  STEM  2,  "High  AOA  Lateral  Gross  Acquisition"  -  STEM  3,  "Roliing 
Defense"  -  STEM  5,  "High  AOA  LongiUidinal  Gross  Acquisition"  -  STEM  10,  "I-g  Stabilized 
Pushover"  -  STEM  16,  and  "Maximum  Performance  Turn  Entry")  were  flown  and  evaluated 
for  the  ability  to  use  them  in  a  flight  tesi  environment. The  six  maneuvers  were  flown 
with  generally  good  success  except  for  the  "Maximum  Perfoimaiice  Turn  Entry"  (which  has 
since  been  eliminated  from  consideration  for  S'lEMS).  Some  of  the  maneuvers  had  to  be 
flown  at  lower  AOA  than  the  ma  cuvers  v'c.re  really  interiued  for  because  of  aircraft  1:  limitations. 
Therefore,  it  would  be  very  beneficial  to  validate  the  STEMS  maneuven:  on  an  aircraft  with 
high  AOA  capability.  Chjrrently,  the  NASA  F-18  HARV  would  probably  he  the  best  aircraft  to 
use  for  a  STEMS  validation  flight  test  program  because  of  its  high  AOA  abilitie.«.  Other  aircraft 
such  as  the  F-16  MATv'  (Multi-Axis  Thrust  Vectoring),  X-31,  or  potentially  the  NASA  F-’5 
3/MTD  ACnVE  (Advanced  Controls  Technologies  for  Integrated  Vehicles)  researr.h  aircraft 
arc  also  capable  of  completing  a  satisfactory  STEMS  validation  fli.ght  test  program. 


Some  of  the  maneuvers  ("High  AOA  Tracking"  -  STEM  2,  'T-Iigli  AOA  Lateral  Gross 
Acquisition"  -  STEM  3,  "High  AOA  Longitudinal  Gross  Acquisition’  •  STEM  10,  and  "1-g 
Stabilized  Pushover"  -  STEM  16)  have  been  or  will  stjon  be  flo’wn  on  the  F-IS  HAR'/.  STEM 
16  has  also  been  flown  on  a  production  F-18-  These  maneuvei.s  have  pio»  ed  useful  for  in¬ 
flight  testing.  However,  it  would  still  be  valuable  to  include  these  maneuver .s  into  a  STEMS 
validation  flight  test  program  so  tliat  they  can  be  directly  compared  ic  all  of  the  STEMS 
maneuvers. 

A  preliminary  flight  test  plan  was  developed,  and  tested  ^tnder  this  conuaci  to  help  nan:>iiion 
the  maneuvers  to  a  flight  test  program  It  can  be  found  in  Reference  5.  The  plan  is  ncf.  meant 
to  be  a  final  version.  Instead,  it  is  intended  to  be  an  intermediate  step  between  the  rriiansuv 
development  conducted  in  fixed-base  simulations  and  the  validation  of  the  maneuvers  in  flight 
test.  The  plan  is  wjiiten  somewhat  gencrically  so  that  it  can  be  used  as  a  starting  point  for  any 
aircraft  evaluation  that  uses  the  STEMS  maneuvers.  However,  it  dfx. .  inc-lude  some  specific 
liARV  data  to  help  initiate  a  validation  test  program  on  the  K.ARV.  Hov  Tver,  it  is 
recomiiiended  that  additional  simulation  with  a  higher  fidelity  simulation  moiicl  be  conducted 
prior  to  flight. 
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A  few  hours  of  simulation  was  conducted  to  evaluate  and  refine  die  flight  test  plan.  A 
simplified  HARV  model  was  used  to  determine  if  any  maneuver  setups  had  to  be  altered 
because  of  the  performance  differcr.ces  between  the  HARV  and  the  generic  configuration  used 
to  develop  the  maneuvers.  The  simulation  was  also  conducted  to  help  detennined  approximate 
maneuver  diuation,  setup  times,  and  energy  lost  during  the  maneuver.  Maneuver  setups  were 
also  altered,  if  necessary  to  allow  the  HARV  to  remain  in  the  Research  Flight  Conaol  System 
(RFCS)  envelope,  thus  allowing  the  use  of  pitch  and  yaw  vectoring.^^  Simulation  data  and 
pre\'ious  flight  test  experience  were  used  to  help  estimate  the  flight  time  required  to  validate  the 
STEMS. 


A 
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Chapter  5 

Summary  of  Design  Parameter  Variation  Data 


As  described  in  Chapters  2  and  4.  several  pieces  of  simulation  data  were  revic;wed  to 
analyze  the  potential  applicability  of  each  maneuver  to  the  design  process.  This  data  included 
time  history  measures  of  merit,  pilot  comments  and  ratings,  responses  to  simulation  comment 
cards,  and  other  information.  Summaries  of  some  of  the  typical  measure  of  merit  results  for 
each  maneuver  are  presented  in  this  chapter.  A  complete  listing  of  the  data  gathered  during  the 
maneuver  testing  is  included  in  Reference  4  As  described  earlier,  these  measuies  of  merit 
were  analyzed  using  statistical  analyses  of  the  simulation  configurations,  and  the  procedure 
outlined  in  Chapter  4  was  used  to  evaluate  the  measures  of  merit.  The  results  obtained  were 
dependent  upon  the  design  parameters  selected  and  the  ranges  of  variation  tested.  Therefore, 
this  is  not  meant  to  be  a  recommendation  for  any  specific  set  of  measures  of  merit  or  design 
parameters. 

A  brief  overview  of  the  maneuvers  and  design  parameters  tested  under  this  research  is 
shown  in  Figure  42.  The  design  parameters  tested  during  tliis  research  were  detailed  in 
Chapter  3,  Figure  22.  Figure  8,  Chapter  1,  should  be  consulted  for  additional  maneuver 
characteristics  such  as  the  axis  being  evaluated,  appropriate  flight  envelope,  etc.  Each 
maneuver  that  was  accepted  as  a  STEM  will  be  briefly  described  in  this  chapter  and  some 
typical  measure  of  merit  results  will  be  shown.  Reference  3  contains  the  complete  maneuver 
description  and  Reference  4  contains  a  full  set  of  data  including  the  pilot  comments.  Only  the 
measures  of  merit  that  resulted  in  reasonably  good  success  will  be  shown  in  this  section.  Also, 
measures  of  merit  were  not  calculated  for  all  of  the  maneuvers  because  it  was  observed  after  the 
second  simulation  that  certain  types  of  maneuvers  were  not  structured  properly  for  repeatable 
numerical  mcasurands.  For  instance,  measures  of  merit  were  not  calculated  for  all  of  the 
freestyle  or  tracking  maneuvers  because  of  the  limited  success  observed  from  the  fu'si  two 
simulations. 
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'J 

V 

17. 

J-Tum 

V 

V 
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V 

V 

18. 

Tanker  Doom  Tracking 

V 

V 

v 

>/ 

19. 

Tracking  In  Power  Approach 

V 

V 

V 

< 

V 

20. 

Offset  Approach  to  Landing 

V 

V 

V 

_ 

V 

ll 

N 

V  Ofisign  Parameter  Successfully  Tested 
N  Design  Parameter  Not  Successfully  Tested 

t  Longitudinal  Dynamics  Indicates  a  Combinatlort  of  Frequency  and  Damping  Tested;  Lateral  Dynamics 
Indicates  a  Combination  of  Roll  Mode  Time  Constant  arwJ  Meudmum  Roll  Rata 

Figura  42.  Design  Par^mstars  Evaluated  With  initial  STEMS  Maneuvers 


STEM  1:  Tracking  During  High  AOA  Sweep 


This  maneuver  is  initiated  with  the  evaluation  pilot  in  trail  of  a  cooperative  target  aircraft 
The  target  enters  a  turn  and  the  evaluation  pilot  evaluates  his  ability  to  track  the  target  The 
maneuver  is  designed  such  that  the  evaluation  pilot  must  gradually  increase  AOA  until  tracking 
can  no  longer  be  conducted.  This  maneuver  has  a  strong  link  to  operational  requirements  and 
is  a  direct  extension  of  the  HQDT^  technique  to  hi^  AOA.  It  can  be  used  to  quickly  evaluate 
longitudinal,  lateral,  and  directional  precision  flying  qualities  over  a  wide  AOA  range  and 
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identify  potential  problem  areas.  If  any  problems  are  uncovered,  then  the  High  AOA  Tracking 
maneuver  (STEM  2)  can  be  used  to  Isolate  an  AOA  for  closer  investigation.  Variations  in  shon 
period  frequency,  short  period  damping,  maximum  roll  rate  (roll  sensitivity),  and  roll  mode 
time  constant  were  tested  to  establish  a  sensitivity  to  design  parameter  modirications. 
Ouantitative  measures  of  merit  were  calculated  but  the  results  were  so  dominated  by  pilot 
variability  that  none  were  successful.  The  measures  of  merit  arc  not  shown  here  bat  are  fully 
documented  in  Reference  4.  Additional  measures  of  merit  could  be  investigated,  but  this 
maneuver  appears  to  be  best  suited  for  qualitative  data.  The  pilot  comments  and  (looper- 
Harper  Ratings  were  of  good  quality  and  are  shown  in  Reference  4. 

STEM  2:  High  AOA  Tracking 

During  this  task,  the  evaluation  pilot  tracks  a  cooperative  target  aircraft  in  a  turn.  The 
maneuver  is  set  up  such  that  the  evaluates  can  maintain  a  relatively  constant  AOA  to  thoroughly 
evaluate  the  tracking  at  that  AOA.  This  maneuver  is  intended  to  expose  air-fc  air  tracking 
flying  qualities  characteristics  at  high  AOA  for  a  single  axis.  A  combLiutlon  of  precise  tracking 
and  small  aim  point  conections  are  used  to  evaluate  tracking  at  a  specific  AOA.  This  maneuver 
was  developed  and  tested  under  MDA  Internal  Research  and  Development2b3234  (iraD)  and 
NASA  sponsored '3.33  ‘-.igh  AOA  flyitig  quaiiries  criteria  development  efforts,  it  is  included  as 
one  of  tlie  initial  STEMS  maneuvers  because  of  its  applicability  to  high  AOA  and  the  fact  that  it 
is  a  relatively  newly  develop.ti  maneuver.  This  task  has  been  used  to  develop  longitudinal  and 
lateral  u-acldng  criteria  for  variations  in  short  period  frequency,  short  period  damping, 
maximum  roll  rate  (roll  sensitivity),  and  roll  mode  time  constant  at  30’,  45*,  and  60*  AOA. 
Design  parameter  variauons  were  not  conducted  under  the  STCMS  contract.  References  18, 

20, 33,  and  34  should  be  consulted  for  specific  pilot  comiiK  nls  and  ratings  obtained  v/ith  this 
task.  In  general,  this  task  appears  to  be  best  suited  for  qualj.ative  flying  qualities  data. 

STEM  3:  High  AOA  Lateral  Gross  Acquisition 

This  maneuver  is  set  up  such  that  the  evaluation  pilot  can  pull  to  a  desired  AOA,  stabilize, 
and  then  roll  to  acquire  a  target  aircraft.  It  was  developed  to  help  isolate  the  lateral  axis  for 
flying  quaiiries  evaluations  at  high  AOA.  Specifically,  die  controllability  of  the  capture  and  the 
roll  rate  achieved  can  be  evalua  ted.  This  maneuver  was  developed  and  extensively  tested  under 
MDA20.32.34  and  NASA  sponscired'833  research.  It  is  included  as  one  o^'  the  initial  STEMS 
maneuvers  because  of  its  applicability  to  high  AOA  and  the  fact  that  it  is  a  relatively  n  ewly 
developed  maneuver,  litis  task  has  been  used  to  develop  lateral  acquisition  criteria  for 
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variations  in  maximum  roll  rate  (roll  sensitixity)  and  roll  mode  time  coiistant  for  30’,  45*,  and 
60*  AOA.  Desigti  parameter  variations  were  not  conducted  under  the  STEMS  contract 
References  18,  20,  and  33  should  be  consulted  for  specific  pilot  comments,  ratings,  and 
criteria  obtained  with  this  task.  Reference  33  also  includes  measure  of  nterit  analyses 
conducted  with  this  task.  This  task  produces  good  fiying  qualities  comments  and  data  as  well 
as  providing  some  reasonably  good  quantitative  information. 

STEM  4:  Dual  Attack 


This  maneuver  consists  of  the  evaluation  aircraft  and  two  target  aircraft.  The  targets  fly 
straight  and  level  with  a  90"  heading  difference  and  the  evaluation  aircraft  maneuvers  between 
them  to  alternately  acquire  each  target  The  pilot  cr  n  evaluate  loaded  roll  capabilities  as  well  as 
the  ability  to  unload,  roll,  and  pull  to  transition  between  the  targets,  'fhis  maneuver  is  an 
operationally  relevant  task  that  highlights  the  ability  to  reach  high  AOA  and  subsequently 
control  the  aircraft  The  advantages  of  good  high  AOA  roll  performance  can  be  demonstrated 
through  this  maneuver.  Variations  in  longimdinal  dynamics,  lateral  dynamics,  and  maximum 
AOA  were  evaluated  with  the  generic  fighter  simulation  nxxlel.  The  MuSIC  model  was  also 
used  to  compare  PST  on  and  PST  off  modes.  A  typical  summary  of  the  measure  of  merit 
results  front  the  generic  fighter  testing  are  showr.  in  Fig;irc  43,  The  variation  in  longitudirial 
dynamics  (LONDYN)  consisted  of  simultaneous  variatiens  in  shc«l  period  frequency  and 
damping,  and  the  variations  in  lateral  dynamics  (LATDYN)  were  tested  by  simultaneously 
varying  maxuiuun  roll  rate  and  roll  mode  time  constant  Maximum  angle  of  attack  (AOAMAX) 
was  varied  between  40*  and  60*  to  produce  the  results  in  Figure  43,  Additional  statistical  tests 
were  conducted  to  determine  the  significance  of  flying  a  loaded  roll  versus  an  unloaded  roll 
technique  (LOAD)  and  calculating  the  measures  of  merit  relative  to  the  first  or  second  target 
(TARGET).  Only  a  few  measures  of  merit  were  found  to  measure  the  variation  in  design 
parameters  successfully  while  being  insensitive  to  pilot  vaiiability.  However,  pilot  comments 
indicate  that  this  is  a  very  good  maneuver  to  evaluate  and  demonstrate  the  benefits  of  high  AOA 
pitch  and  roll  capabilities. 
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Strong 

Potentialiy  Strong 
Potentially  Poor 
Poor 


LONDYN  AOAMAX  LATDYN  LOAD  TARGET 


Time  of  Max  Lift  Coefficient 
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Figure  43.  Overall  SenaitlvKlM  lor  Duel  Attack  (STEM  4  TEST  1  ANALYSIS  6} 


STEM  5:  Rolling  Defense 


To  set  up  this  maneuver,  the  pilot  establishes  a  turn  at  the  desired  test  conditions  (AOA  and 
airspeed,  or  load  factor)  and  initiates  a  full  stick  roll  over  the  top.  The  data  taking  portion  of 
the  maneuver  begins  when  the  pilot  reaches  the  opposite  90*  bank  angle  and  applies  a  full 
forward  stick  input  while  maintaining  lateral  stick.  The  maneuver  is  tenninated  when  the 
aircraft  has  unloaded.  Hiis  maneuver  is  primarily  intended  as  a  control  law  evaluation 
maneuver  to  verify  the  nose-down  pitch  authority  remaining  while  in  a  rolling  condition. 
Additional  infesmatien  about  roll  coordination  and  maximum  roll  rate  may  also  be  obtained. 
Variations  in  roaximum  roll  rate,  center  of  gravity  location,  and  pitch  vectoring  were  tested 
using  this  maneuver.  Figure  44  shows  a  suixunary  of  the  measure  of  merit  i&sults  £tom  tite 
maximum  roll  rate  (PMAX)  and  center  of  gravity  (DCG)  variations.  Several  measures  of  merit 
were  found  to  be  sensitive  to  each  of  these  design  parameters.  This  indicates  good  rqieatability 
in  the  maneuver  and  indicates  that  numerical  data  frenn  this  maneuver  can  be  used  to  nxxUfy  the 
design.  Some  pilot  comments  were  also  gcnerafcd  frcmi  this  maneuver,  but  it  tended  to  be  vety 
dynamic  in  nature  and  somewhat  difricult  to  ctnnment  on. 
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Figure  44.  Overall  Sensltlvltlas  for  Rolling  Oafanaa  (STEM  5  TEST  2) 


STEM  6:  Matsimum  Pitch  Pull 


This  maneuver  is  very  simple  in  that  the  pilot  establishes  the  desired  trim  tes:  point  and  then 
applies  an  aggressive  full  aft  stick  input  and  holds  it  until  the  pitch  rate  had  stopped.  This 
nnancuver  represents  a  fundamental  element  of  several  maneuvers  by  isolating  an  aggressive, 
open-loop  longitudinal  input.  Testing  was  conducted  at  very  low  speed  (Vmjn)  and  at  comer 
aii^pccd  (Vc).  Variatiors  in  short  period  frequency  (WSP),  short  period  damping  (ZSP),  and 
maximum  uiglc  of  attack  (AOAMAX)  were  tested  for  the  low  speed  case.  Figures  45  and  46 
show  typical  results  for  this  testing  and  illustrate  an  impemant  consideration  when  conducting 
measure  of  merit  screening.  Different  indications  of  the  sensitivity  to  design  parameters  were 
obtained  between  these  figures  because  of  the  different  range  of  AOAMAX  tested.  The  range 
of  WSP  and  ZSP  for  these  tv/o  tests  was  identical,  hit  the  results  in  Figure  45  occurred  when 
AOAMAX  was  varied  between  40’  and  70’  whereas  the  res'olts  in  Figure  46  are  based  on  a 
variation  between  40’  and  55*.  The  large  variation  in  AOAMAX  used  to  produce  Figure  45 
tended  to  dcirdnate  the  results.  It  resulted  in  a  large  number  of  black  region*,  for  the  AOAMAX 
design  parameter  and  reduced  the  strength  of  correlation  for  the  other  two  design  parameters. 
This  example  illustrates  the  fact  that  the  tnea.sure  of  merit  findings  are  dependent  upon  the 
ranges  tested,  and  that  care  must  be  taken  when  using  DOE  techniques  so  that  the  test  matrix  is 
balanced  Kowever  in  general,  dre  maximum  pitch  pull  maneuver  was  useful  in  gathering 
quantitative  dau.  because  i^  is  a  very  simple,  repeatable  maneuver  that  isolates  the  pitch  axis. 
Some  pilot  comments  can  also  be  obtained  but  it  is  not  useful  for  flying  qualities  development 
because  it  is  an  q}en'loop  maneuver. 
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Figure  45.  Overall  Sensltlvitlas  for  Max  Pitch  Pull  (STEM  6  TEST  1  ANALYSIS  A) 
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Figure  46.  Overall  Sensitivities  lor  Max  Pitch  Pull  (STEM  6  TEST  1  ANALYSIS  B) 


The  maximum  pitch  pull  maneuver  was  tested  at  comer  airspeed  also.  The  maneuver  was 
initialed  fiom  a  dive  to  allow  a  greater  pitch  angle  change  to  be  tested-  Variations  in  Control 
Anticipation  Parameter  (CAP),  shon  period  damping  (ZSP),  and  maxiinum  attainable  load 
factor  (NZMAX)  were  tested  for  the  comer  airspeed  conditio:  Figure  47  shows  typical 
nieasure  of  merit  results  for  the  higher  speed  case.  Just  as  with  the  low  speed  veision, 
quantitative  data  could  be  used  to  tie  measures  of  merit  back  to  design  parameters,  and  some 
pilot  comments  were  obtained. 
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Figure  47.  Overall  Senaltivltlea  for  Maximum  Pitch  Puli  (STEM  6  TEST  3) 


STEM  7:  Nose-Up  Pitch  Angle  Capture 


This  maneuver  setup  is  the  same  as  STEM  6  and  the  inaxinium  pitch  attitude  that  can  be 
captured  is  determined  from  STEM  6.  A  target  aircraft  is  positioned  ahead  of  and  above  die 
evaluation  aircraft  to  provide  a  reference  to  capture.  This  maneuver  represents  a  fundamental 
element  of  several  maneuvers  by  isolating  an  aggressive  longirudinal  capture  task.  Testing  was 
conducted  at  very  low  sp  i  (Vmin)  and  at  comer  airspeed  (Vg).  The  low  airspeed  setup  was 
tested  with  variations  in  s*.ort  period  frequency  (WSP),  short  period  damping  (ZSP), 
longitudinal  stick  sensitivity  (LONSNS),  time  delay  (TIMDEL),  and  nonlinear  stick  shaping 
(LONSHP).  TTie  few  measures  of  merit  and  design  parameters  that  resulted  in  successful 
correlation  are  shown  in  Figure  48.  Most  of  the  measures  of  merit  were  dominated  by  pilot 
variability  because  of  the  closed-loop  nature  of  the  task.  Time  to  capture  was  also  calculated, 
but  it  was  not  effective  because  of  the  large  amount  of  pilot  technique  required  in  this 
maneuver.  The  LONSHP  design  parameter  did  not  generate  any  successful  measure  of  merit 
correlations  so  it  is  not  shown  in  Figure  48.  However,  tliis  maneuver  did  generate  very 
effective  pilot  comment  data  and  Cooper-Harper  ratings.  All  of  the  design  parameters  except 
for  TIMDEL  could  be  evaluated  using  pitot  comment  data.  Historically  it  has  been  difficult  to 
determine  accurately  the  effects  of  moderate  time  delays  in  a  fixed-base  simulation. 


The  nose-up  pitch  angle  capture  maneuver  was  tested  at  comer  airspeed  also.  The 
maneuver  was  initiated  from  a  dive  to  allow  a  greater  pitch  angle  change  to  be  tested. 

Variations  in  Control  Anticipation  Parameter  (CAP),  short  period  damping  (ZSP),  and 
longitudinal  stick  shaping  (LONSHP)  were  tested  for  tlie  comer  airspeed  condition.  Figure  49 
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shows  typical  measure  of  i.:crit  results  for  ihc  higher  speed  case.  Just  as  nith  the  low  speed 
veifion,  very  little  quantitative  data  could  be  used  to  tie  measures  of  merit  back  to  design 
parameters.  However,  the  time  to  capture  metric  proved  more  reliable  for  the  higher  speed  test 
condition.  Pilot  comments  and  ratings  were  again  very  valuable  for  the  comer  airspeed  test 
condition. 
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Figure  48.  Overall  Sensitivities  for  Nose-Up  Pitch  Angle  Capture  (STEM  7  TEST  7 

ANALYSIS  D) 
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Figure  49.  Overall  Sensitivities  for  Nose-Up  Pitch  Angle  Capture  (STEM  7  TEST  6) 


STEM  8:  Crossing  Target  Acquisition  and  Tracking 


This  maneuver  is  conducted  by  setting  up  a  target  aircraft  above  the  evaluation  aircraft  with 
a  90*  heading  offset,  and  the  target  immediately  enters  a  turn  toward  the  evaluation  aircraft. 
The  evaluation  pilot  tries  to  acquire  rapidly  and  then  continue  fracking  the  target  as  it  crosses  in 
front.  This  maneuver  allows  the  acquisition  and  tracking  capabilities  of  an  aircraft  to  be 
exercised  through  the  multiple-axis  acquisition  of  a  target  aircraft.  The  maneuver  requires  tlie 
test  aircraft  to  generate  and  stop  a  pitch  rate  to  capture  the  utrget,  as  well  as  perform  a  multiple 
axis  tracking  task  on  a  crossing  target.  The  ability  to  pull  to  mioderately  high  AOA,  stop  the 
pitch  rate,  laterally  track  a  target  while  unloading  in  AOA,  and  tl.en  transition  to  longitudinal 
tracking  are  tested.  This  maneuvei  was  used  to  test  variations  in  short  period  frequency, 
maximum  roll  rate,  and  roll  mode  time  consfanL  Valuable  pilot  comments  on  longitudinal 
flying  qualities,  lateral  flying  qualities,  and  control  harmony  were  obtained  from  this 
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maneuyta-.  However,  quantitative  tneasurts  of  rumt  were  not  calculated  because  of  the 
fireestyle  nature  of  the  task. 


JTEM  9:  Pitch  Rate  Reserve 


This  maneuver  is  conducted  by  establishing  a  level  turn  at  die  desired  test  conditions  and 
applying  a  full  aft  stick  input  The  input  is  maintained  until  the  nose  rate  drops  below  its  initial 
stabilized  value.  This  maneuver  is  intended  to  demonstrate  the  reserve  pitch  authority  available 
from  a  loaded  condition.  This  maneuver  was  defined  from  the  "Angular  Reserve"  maneuver 
tested  ir.  Reference  6.  Design  parameter  variations  in  short  jjxriod  fretiuency  (WSP),  shtHt 
period  damning  (ZSP),  and  longitudinal  stick  sensitivity  (LONSNS)  were  tested.  Figure  50 
shows  results  of  the  nv.  isure  of  merit  analysis  conducted  for  this  maneuver.  Several  measures 
of  merit  successi  ally  correlated  the  sknuladon  data  indicating  the  ability  to  use  quantitauve  data 
for  this  maneuver.  The  design  pararejeter  LONSNS  did  not  generate  significant  changes  in  any 
cf  the  calculated  measures  of  merit.  Sonv;  valuable  pilot  comment  data  was  obtained  using  tliis 
maneuver,  but  it  was  limited  because  of  the  open-loop  nature  of  the  maneuver. 
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FIgur*  90.  Ovemll  Scn»ltlvltl««  for  Pitch  Rato  Rasorv*  (STEM  6  TEST  2) 


STEM  10:  High  .kOA  Longitudinal  Gross  Acquisition 


Thi':  inaneuver  is  initiated  with  the  evaluation  aircraft  in  trail  of  a  target  aircraft.  The  taiget 
enters  a  turn  and  the  evaluation  pilot  perfenos  a  sequence  of  longitudinal  acquisitions  of  the 
target  aircraft.  With  seme  practice,  the  pilot  can  perform  the  capioirs  so  that  they  occur  around 
a  test  \OA.  'rhis  maneuver  is  designed  to  isolate  tfic  flying  qualities  characteristics  of  an 
aircraft  daring  a  high  AOA  longitudinal  capnne  tasL  It  was  d&vdqpcd  and  tested  under  MDA 
IllAD^'3234  an(j  NASA  sponsored*^*^^  high  AOA  flying  qualities  critenn  dcvclojimfiU 


effom.  It  is  included  as  one  of  the  initial  STEMS  maneuvers  because  of  its  applicability  to 
high  AOA  and  the  fact  that  it  is  a  relatively  newly  developed  ma.ieuver.  This  task  has  been 
used  to  develop  longitudinal  acquisition  crite.ia  for  variations  in  short  period  frequency  (WSP) 
and  short  pcritxl  damping  (ZSP)  for  severai  angles  of  attack.  A  minimal  amount  of  data  was 
taken  under  the  STEMS  contract.  References  18, 20, 33,  and  34  should  be  consulted  for 
additional  pilot  comments;  ratings,  and  flying  qualities  criteria  obtained  from  this  maneuver. 
Reference  33  also  includes  measure  of  merit  analyses  conducted  with  data  ftxMn  this  maneuver. 
A  summary  of  the  measure  of  merit  analysis  conducted  under  the  STEMS  contraa  is  shovtTi  in 
Figure  51.  The  few  number  of  successful  correlations  in  Figure  51  may  be  due  to  the  minimal 
amount  of  data  collected;  Reference  33  was  more  successful  at  extracting  meaningful  measiue 
of  merit  data  from  this  maneuver.  Overall,  this  task  produces  good  flying  qualities  comments 
and  data  as  well  as  providing  some  reasonably  good  quantitative  information. 
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Flgurft  SI.  Overati  Senaltivltlat  for  High  AOA  Longitudinal  Groaa  Aequisltion 


{STEM  10) 


STEM  11:  Sharkenhausen 


This  task  is  initiated  witli  a  head-on  target  aircraft  that  is  downrange,  offset,  and  higher  than 
the  evaluation  aircraft.  The  evaluation  pilot  attempts  to  capture  the  target  as  rapidly  as  possible 
and  then  track  the  target.  This  maneuver  ?Uowr.  the  acquisition  capabilities  of  an  aircraft  to  be 
exercised  through  a  multiple-axis  acquisition  of  a  target  aircraft.  The  ability  to  pull  to 
moderately  high  AOA  and  maintain  good  lateral  control  on  a  crossing  target  is  emphasized. 
Ihis  mancuva  /as  evaluated  at  a  low  speed  condition  (Vmin)  and  at  comer  airspeed  (Vc). 
Design  parameters  variations  in  longirodinal  dynaiiucs  (LONDYN),  lateral  dynanucs 
(LATDYN),  and  nuiximum  angle  of  attack  (AOAMAX)  were  tested  for  the  low  speed 
condition.  The  variation  in  longitudinal  dynamics  consisted  of  simultaneous  variations  in  shon 
period  frequency  and  damping,  and  the  variations  in  lateral  dynartucs  were  tested  by 
simultaneously  varying  maximum  roll  rate  aird  roll  mode  time  constant  A  quick  investigation 
of  initial  range  to  target  was  also  tested.  A  typical  summary  of  the  measure  of  merit  data  is 
shown  in  Figure  52.  Some  measures  of  merit  were  found  to  correlate  to  these  design 
parameters;  however,  the  data  is  very  dependent  upon  initial  range.  TTiis  implies  that  the  task 
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may  be  limited  to  simulation  use.  It  does  appear  to  be  a  valuable  evaluation  and  demonstration 
maneuver  however.  The  initial  downrangc  could  also  be  vaiied  to  determine  the  minimum 
range  at  which  the  task  could  be  performed.  Design  parameters  could  then  be  varied  to 
determine  their  influence  on  this  minimum  range. 
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Figure  32.  Overall  Sentttivttlee  SharkenhauMn  (STEM  ii  TEST  5} 


Variations  in  LONDYN,  LATDYN,  and  AOAMAX  were  also  tested  with  the 
Sharkenhausen  at  comer  airspeed.  A  summary  of  the  measure  of  merit  analysis  is  shown  in 
Figure  53.  'fhe  high  speed  test  condition  scented  to  be  more  dependent  upcm  pilot  technique 
because  of  the  greater  load  factor  capability.  Very  few  strong  correlations  were  observed  and 
LONDYN  resulted  in  no  successful  correlations.  Overall,  the  initial  range  to  target  had  a 
stronger  influence  <m  the  measures  of  merit  than  did  any  of  the  design  parameters  tested.  As  a 
result,  poor  quantitative  data  resulted  from  this  maneuver.  However,  this  maneuver  produced 
qualitative  data  that  ca.i  he  used  as  an  overall  check  of  die  aircraft's  ability  to  point  at  and  track 
a  crossing  target  rapidly. 
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Figure  53.  Overail  Sensitivities  for  Sharkenhausen  (STEM  ii  TEST  4) 


STEM  12:  High  AOA  Roll  Reversal 


Setup  for  this  maneuver  is  accomplished  by  performing  a  split-S,  then  pulling  lo  the  test 
AOA.  The  data  gathering  portion  of  the  maneuver  begins  aftet  the  pitch  attitude  increases  to  the 
point  that  the  velocity  vector  is  pointed  directly  downward.  At  dial  time,  the  pilot  applies  a  full 
roll  control  input  and  holds  it  until  the  heading  has  changed  by  the  desired  anxiunt.  The  pilot 
then  applies  full  opposite  roll  controls  until  returning  through  the  initial  heading.  This 
maneuver  allows  the  inve.stigation  of  high  AOA  roll  perfcHtna.nce  Ln  a  reladv  '^ely  stabtizzed  flight 
condition.  Roll  onset  as  well  as  the  airoraft  response  to  a  large  cross-check  input  can  be 
evaluated.  This  maneuver  was  originally  suggested  in  Reference  6  and  developed  and  tested 
under  this  research.  This  maneuver  was  flown  using  90'  and  180’  heading  changes. 

Variations  in  roll  mode  time  constant  (TR),  maximum  roll  rate  (PMAX),  and  w  hether  or  not  a 
roll  acceleration  limiter  existed  (PDLIM)  were  tested.  Figure  54  includes  a  summary  of  the 
overall  measure  of  merit  results  for  the  180*  heading  change.  Maximuin  roll  rate  was  the 
predominant  design  parameter  in  this  case  and  PDLIM  did  not  produce  any  strong  cOTrelation. 
Roll  mode  time  constant  and  the  roll  acceleration  limit  were  more  important  when  the  maneuver 
was  flown  tlirough  only  a  90’  heading  change  as  seen  in  Figure  55.  Maxunum  roll  rate  is  still 
a  stronger  design  parameter,  but  roll  mode  tinr  constant  and  the  roll  acceleration  limit  are  now 
significant  This  may  be  attributed  to  the  tact  that  the  initial  response  is  a  much  larger  portion 
of  the  90’  maneuver  than  the  180’  maneuver.  Tnerefore,  parameters  that  most  affect  the  initial 
response  become  more  of  an  influence.  Overall,  this  maneuver  was  more  effective  at 
generating  numerical  data  than  pilot  comments,  but  some  useful  comments  were  obtained. 
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Figure  54.  Overall  SenaHIvltles  for  High  AOA  Roil  Reversal  (STEM  12  TEST  1) 
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Figure  55.  Overall  SenaltivKies  for  High  AOA  Roll  Reversal  (STEM  12  TEST  2) 
STEM  13:  High  AOA  Roll  end  C:  pture 

This  maneuver  is  initialed  similarly  to  STEM  12,  but  the  pilot  perfemns  a  lateral  capture 
instead  of  using  full  opposite  roll  controls  to  reverse  the  roll.  The  pilot  can  capture  a  heading 
because  of  the  velocity  vector  is  oriented  straight  down  during  this  maneuver.  This  maneuver 
is  designed  to  isolate  the  flying  qualities  chanaaehstics  of  an  aimaft  during  a  high  AOA  lateral 
capture  task.  Figure  56  shows  the  ir<easure  of  merit  analysis  /or  variations  in  roll  mode  dnte 
constant  (TR)  and  maximum  roll  rate  (PMAX).  Data  is  only  available  for  one  pilot,  so  the 
correlations  on  this  figure  do  not  factor  in  pilot  variability.  As  a  result,  a  final  conclusion  on 
the  strength  of  measures  of  merit  cannot  be  determined  yet,  but  the  initial  indications  look 
promising.  The  pilot  comment  and  rating  data  received  from  this  maneuver  appear  to  be 
valuable. 
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Note;  Pilot  Variability  Not  Included  Because  Data  Available  from  Only  One  Pilot 

Figure  56.  Sensitivities  to  Design  Parameters  for  High  AOA  Roll  and  Capture 

(STEM  13) 


STEM  14:  Minimum  Speed  Full  Stick  Loop 


This  maneuver  is  flown  iteratively  in  a  buiid-up  fashion  to  idenb/y  an  airspeed  band  in 
which  a  full-stick  loop  cannot  be  completed.  The  maneuver  is  started  at  a  low  sp'^ed  imd  a 
maximum  pitch  pull  is  performed.  The  start  speed  is  successively  increased  until  an  80*  pilch 
attitude  is  attained  'The  maneuver  is  then  attempted  at  100  knots  faster  than  the  speed  reqinred 
to  reach  80*  pitch  attitude.  This  stait  speed  is  then  successively  reduced  until  the  minimum 
pitch  rate  drops  below  5  deg/sec  or  lateral  control  becomes  deficient.  Inform;mor.  on  pitch 
authonty  at  low  speeds  in  the  vertical  as  well  as  roll  stability  information  may  be  obtained.  It 
does  not  represent  the  minimum  airspeed  at  which  a  loop  can  be  flown  using  energy- 
maneuverability  principles.  Thus  maneuver  tends  to  be  mo.te  of  a  demonsttation  ?jid  envelope 
expansion  maneuver  rather  than  a  design  evaluation  maneaver.  ’Therefoit;,  the  only  variation 
tested  was  with  the  MuSIC  aircraft  with  PST  on  raid  PST  off.  No  quantitative  data  is  intended 
to  be  calculated  for  this  maneuver  other  than  the  minimijm  speed  fo!’  a  full  stick  loop.  It  did 
result  in  pilot  comments  about  low  speed  contKdiabilin . 

STEM  15:  Minimum  Time  lf»0'  Heading  Change 


This  maneuvei  is  in»er..1ed  to  demonsirat'.:  the  possible  options  that  a  pilot  has  aviiilaWc  to 
change  the  air.vafi  heading  by  180“.  It  shojid  include  ’.esting  of  conventional  methods  such  a.; 
level  tun-s,  the  spIit-S,  slices,  as  well  as  techniques  such  as  the  J  Tian.  Only  the  initiid  and 
final  conditions  are  specified  for  this  maneuver.  If  is  a  freestyle  maneuver  because  the  pilot  is 
encouraged  to  try  various  techniques  to  treiform  a  180“  tseading  change.  'Hus  maneuvei  was 
flown  vrith  the  Mu8lC  simulation  model  using  the  PST  on  and  off  modes  to  de.monstn>te  Uie 
ad'iiL'ona’  options  provided  to  a  pilot  thiough  dirust  vectoring.  'This  maneuver  is  v.ot  inicndoi 
fo:  quantitadve  data  except  for  a  lough  tsiimae  of  umc  required  tr.'  change  heariitig  by  180“. 
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STEM  16  1-g  Stabilized  Pushover 


To  perform  this  maneuver,  the  pilot  establishes  a  stabilized,  wings  level  liigh  AOA 
condition  and  aggressively  applies  full  forward  stick.  The  pilot  continues  to  hold  forward  stick 
until  the  AOA  drops  below  10*.  This  maneuver  allows  a  stabilized  evaluation  of  the  nose- 
down  pitch  authority  at  high  AOA.  This  maneuver  was  developed  and  tested  under 
NASA/Navy  researched,  h  is  included  as  one  of  the  initial  STEMS  maneuvers  because  of  its 
applicability  to  high  AOA  and  the  fact  that  it  is  a  reladveiy  newly  developed  maneuver.  This 
maneuver  was  used  to  test  variations  in  center  of  gravity  location  (DOG)  and  pitch  vectoring 
(TV).  Figure  57  shows  measure  of  merit  results  frosn  the  DCG  testing.  Four  center  of  gravity 
locations  were  tested.  Ihe  column  labeled  DCGA  compares  the  most  forward  eg  location  to 
the  second  most  forward.  The  DCXjB  column  compares  the  most  forward  eg  to  the  third  most 
forward,  and  the  DCGC  column  compares  the  forward-nx)st  and  aft-most  eg  locations.  The 
measures  of  merit  are  quite  good  for  the  DCGA  variation  and  continue  to  become  stronger  as 
the  center  of  gravity  location  moves  further  aft  Tnis  maneuver  generates  very  consistent 
quantitative  data  because  of  its  simple,  repeatable  technique.  Pilot  comments  and  Pitch 
Recovery  Ratings  can  also  be  used  from  this  maneuver. 
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STEM  17  J-I  .  rn 


To  perform  this  maneuver,  the  pilot  applies  full  pitch  and  roll  control  inputs  simultaneously 
until  the  aircraft  has  completed  a  180*  turn  or  reaches  a  wings-level  inverted  positiou, 
whichever  occurs  fin>t.  Then  the  pilot  removes  the  roil  control  input  and  continues  to  pitch  the 
nose  back  up  to  the  horizon.  This  maneuver  requires  the  simultaneous  use  of  high  AOA  pitch 
and  roll  authority  and  serves  as  a  good  demonstration  maneuver  for  high  AOA 
maneuverability.  The  J-Tum  is  intended  to  emulate  the  maneuvering  requirements  of  a  tactic 
dev^eloped  diuing  tlie  MuSIC  thrust  vectoring  tactical  utility  studies.^^  lliis  maoueuver  was 
used  to  investigate  variations  in  short  period  frequency  (WSP),  ma’amum  roll  rate  (PMAX), 
and  longitudinal  command  types  (CMDTYP).  AOA  and  AOA  rate  longitudinal  command  types 
were  compared.  It  was  also  used  to  compare  the  PST  on  and  PST  off  modes  of  the  MuSlC 
simulation  model.  Figure  58  shows  a  summary  of  the  WSP,  PMAX,  and  CMDTYP  testing. 
For  the  measures  of  merit  calculated,  the  variation  in  WSP  resulted  in  the  most  correlations. 
However,  both  CMDTYP  and  PMAX  resulted  in  several  successful  correlations.  This 
maneuver  also  resulted  in  some  pilot  cotninents  but  seems  to  be  best  suited  as  a  maneuver  to 
demonstrate  high  AOA  roil  and  pitch  authority. 

STEM  18;  Tanker  Boom  Tracking 

This  maneuver  consists  of  tracking  the  refueling  probe  of  a  tanker  from  a  pre-contact 
posiaon.  The  evaluation  pilot  can  try  tracking  a  steady  probe  or  the  boom  operator  can  move 
the  probe  to  create  tracking  errors.  Tliis  mai>euver  is  intended  to  evaluate  high  gain  flying 
qualities.  It  will  highlight  high  gain/high  sensitivity  flight  control  system  deficiencies  and 
fiossibly  uncover  low  phase  and  gain  margins.  This  is  an  existing  maneuver  but  was  further 
tested  here  for  validation  and  because  it  may  not  be  a  well  recognized  evaluation  maneuver. 
This  maneuver  is  used  at  tlie  Air  Force  Flight  Test  Center  and  in  particular  was  recently  used 
on  the  C-17  progiam.^5  Variations  ii'  Control  Anticipation  Parameter  (therefore  short  period 
frequency),  sho.  t  i)eriod  damping,  and  roll  mode  time  constant  were  tested  with  the  generic 
fighter  and  transport  models  Ihe  variations  in  dynamics  were  discernible  to  the  pilots  and 
resulted  in  good  comments.  Coopcr-Haiper  and  PIO  ratings  are  also  applicable  for  this 
maneuver.  This  task  was  more  difficult  to  fly  in  tlie  fixed-base  simulatoi  than  might  be 
expected  i.i  flight.  It  appealed  that  PIO  tendencies  were  exaggerated  and  it  was  more  difficult 
to  control  the  range  to  prooe.  'fhese  characteristics  are  attributed  to  the  reduced  pilot  cues  as 
compared  to  flight.  It  is  still  believed  to  bo  a  valuable  task;  however,  fixed-base  simulation 
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may  result  in  an  overly  pessimistic  evaluation.  Numerical  measure  of  merit  analyses  were  not 
attempted  becar.s*^  of  the  tight  closed-loop  nature  of  this  task. 
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Figure  58.  Overall  SensittvHleii  for  J-Turn  (STEM  17  ANALYSES  A  and  B) 


STEM  19;  Tracking  in  PA 


This  maneuver  consists  of  tracking  a  target  aircraft  from  approximately  15(X)  ft  range  while 
in  a  power  approach  mode  and  at  an  approach  airspeed.  It  was  valuable  to  have  the  target 
paforai  a  sequence  of  heading  changes  to  form  a  more  demanding  task.  This  maneuver  is  a 
task  that  can  be  performed  at  a  safe  altitude  before  precision  landings  arc  attempted.  This  was 
an  existing  maneuver  but  was  further  tested  here  for  validation  and  because  it  may  not  be  a  well 
recognized  evaluation  maneuver.  In  particular,  this  noaneuver  was  used  on  the  F- 15  S/MTD 
program^^^  Variations  in  Contiol  Anticipation  Parameter  (therefore  short  period  frequency). 
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shon  period  damping,  maximim^  roil  rate  (roll  sensitivity),  and  roll  mode  time  coristant  were 
tested  with  the  generic  fighter  and  transpon  models.  The  maneuver  resulted  in  reasonably 
valuable  comments  and  ratings  from  the  fighter  testing  but  the  variations  in  dynamics  were  not 
very  discernible  during  the  transjxin  testing.  This  may  have  been  due  to  the  design  parameter 
range  tested  or  the  small  heading  variations  of  the  target  aircraft  (15*  neading  changes  for  ilie 
transpon  task  versus  30*  heading  changes  for  the  fighter  testing).  Measures  of  merit  were  not 
calculated  for  this  task  because  of  the  closed  loop  stature  of  the  task.  Additional  testing  and 
validation  of  the  this  maneuver  is  recommended;  however,  it  appears  to  be  a  promising 
maneuver. 


STEM  20  Offset  Approach  to  Landing 


This  task  is  initiated  with  the  airaaft  on  the  correct  glide  slope,  correct  approach  speed,  and 
parallel  to  the  runway  but  offset  to  one  side.  At  a  specified  position,  the  pilot  corrects  the 
lateral  offset  and  attempts  a  prccisicn  landing.  This  maneuver  provides  a  demanding  flying 
qualities  task  to  test  the  ability  to  control  flight  path  and  speed  while  the  aircraft  is  configured 
for  approach.  This  maneuver  has  been  used  extensively  to  evaluate  aircraft  approach  to  landing 
flying  qualities.  Preliminaiy  testing  was  conducted  as  part  of  this  contract  to  inve-srigatc 
variations  in  maximum  roll  rate  (PMAX),  roll  mode  time  constant  (TR),  Control  Anticipation 
Parameter  (CAP),  short  period  damping  (ZSP),  engine  response  time  constant  (TAtJENG), 
time  delay  (TIMDEL),  and  lift  curve  slope  (LALPHA  -  pilch  rate  lead  term).  During  testing, 
the  aircraft  speed  control  was  found  inadequate  and  that  tended  to  dominate  the  pilot  comments. 
At  that  point,  testing  was  suspended  because  of  the  amount  of  data  and  testing  that  has  already 
been  conducted  with  this  task  in  other  research.  Measures  of  merit  were  calculated  from  the 
limited  testing  conducted  under  the  STEMS  contract.  Several  measures  of  merit  were 
attempted,  but  Figure  59  shows  that  few  resulted  in  any  success.  This  figure  also  indicates  that 
only  the  PMAX,  TR,  and  CAP  design  parameters  resulted  in  any  success.  The  nature  of  this 
task  is  such  that  it  may  require  significantly  more  samples  to  produce  a  reliable  statistical 
analysis.  The  pilot  comments  were  difficult  to  analyze  because  of  the  speed  ccntrol  deficiency 
of  the  aircraft  and  the  DOE  test  naatrix  chosen.  fFhc  seven  factor  test  matrix  shown  in  Figure 
26  was  used.)  However,  this  maneuver  is  included  in  STEMS  because  it  has  proven  to  be  a 

valuable  evaluation  icol  in  several  other  rcsearc''  id  dcvelopinetii  programs. 
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Figure  59.  Overall  Scnaltivltios  for  Oftaet  Approach  to  Landing  (STEM  20) 
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Chapter  6 

Concluding  Remarks 


Three  primary  products  resulted  from  this  research.  A  Standard  Evaluation  Maneuver  Set 
was  initiated  by  developing  and  documenting  several  new  maneuvers.  TTiese  maneuvers  are 
designed  to  help  evaluate  an  aircraft  in  an  operational  environment  and  can  be  used  to  evaluate  a 
wide  range  of  aircraft  attributes  and  capabilities.  As  a  result,  the  maneuvers  can  be  used 
throughout  the  design  process  to  produce  an  operationally  effective  aircraft.  These  maneuvers 
have  been  documented  in  a  maneuver  reference  guide  that  is  intended  to  beccane  a  "living" 
reference  source  of  u.seful  evaluation  maneuvers.^  The  maneuvers  have  been  written  in  a 
somewhat  generic  form  so  that  they  can  be  altered  as  necessary  to  meet  the  specific  test 
objectives  for  any  aircraft  program. 

The  second  major  product  of  this  research  is  a  maneuver  development  process  that  can  be 
used  to  define  additional  evaluation  maneuvers.  The  development  of  new  maneuvers  is 
important  because  the  initial  entries  into  STEMS  do  not  provide  a  comprehensive  set  of 
evaluation  maneuvers.  The  process  documented  within  this  report  proved  to  be  an  effective 
method  to  develop  and  evaluate  maneuvers.  The  key  principles  required  to  develop 
maneuvers,  important  characteristics  of  these  maneuvers,  and  lessons  learned  arc  also 
described.  This  maneuver  development  process  can  be  used  to  generate  additional  STEMS  as 
new  technologies  emerge  ot  new  capabilities  are  added. 

The  third  product  of  this  research  is  a  set  of  guidelines  to  help  select  existing  naneuvers.  It 
may  not  be  necessary  to  test  ail  of  the  STEMS  maneuvers  for  a  particular  configuration  or 
design  parameter  trade-off  study.  So,  a  set  of  guidelines  has  been  develtqrcd  to  help  STEMS 
users  select  the  best  subset  of  maneuvers  their  particular  test  needs.  These  guidelines  have 
been  included  with  the  maneuver  reference  guide^  to  create  a  stand-alone  working  document. 

There  are  several  benefits  that  can  be  gained  through  the  use  of  these  three  products.  High 
quality  evaluation  maneuvers  can  be  developed  more  efficiently.  The  time  required  to  test  an 
aircraft  can  be  reduced  by  using  predefined,  wcU-documcntcd  evaluation  maneuvers.  The  time 
required  to  plan  for  a  test  can  be  reduced  and  die  quality  of  the  test  can  be  improved  by  using 
die  maneuver  selection  guidelines.  And  most  hnponantly,  a  mwe  constructive  evaluation  can 
be  conducted  by  evaluating  key  aircraft  attributes  in  operationally  representative  tasks. 

Additional  data  on  each  successful  maneuver  tested  during  this  simulation  is  documented  in 
Reference  4.  This  data  represents  a  large  number  of  test  points  and  may  contain  valuable 


83 


information  for  those  who  want  to  investigate  these  maneuvers  further.  For  example,  the  pilot 
comments,  ratings,  and  measure  of  merit  data  can  be  used  to  help  develop  initial  test  matrices 
for  flying  qualities  criteria  development  or  tactical  utility  studies.  The  data  may  also  provide 
background  data  for  the  development  and  application  of  numerical  measures  of  merit. 
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Chapter  7 
Recommendations 


The  strongest  rccoramendation  for  future  efforts  is  to  continue  the  development  of  new 
mcuieuvers  for  updates  to  the  Standard  Evaluation  Maneuver  Set.  This  research  is  just  the  first 
step  toward  a  maneuver  reference  guide  for  the  evaluation  of  a  wide  range  of  aircraft 
characteristics.  STEMS  will  be  an  extremely  valuable  design  tool  if  additional  maneuvers 
continue  to  be  added.  Evaluation  maneuvers  that  are  developed  in  the  future  fa*  emerging 
technologies  should  be  included  in  this  "living"  document.  It  is  also  recommended  that 
existing,  well  established  maneuvers  be  included  a-s  pan  of  STEMS  so  that  they  can  be  more 
uniformly  documented  and  become  nwrf;  widely  used.  It  is  hoped  that  STEMS  will  be  a 
convenient,  and  therefore  often  used,  sc/urce  of  maneuver  descriptions  and  ideas  that  will  be 
used  to  improve  an  aircraft  design  from  initial  development  in  simulation  to  final  flight  test 
New  maneuvers  or  experience  in  applying  existing  STEMS  maneuvers  should  be  sent  to 
Wright  Laboratory/FIGC_2  for  inclusion  into  STEMS. 

The  initial  set  of  maneuvers  was  developed  in  a  fixed-base  simulation;  thercfOTe,  it  is 
reconiiitciiu  iiiai  in-aight  testing  be  conducted  to  evaluate  these  maneuvers  in  a  flight  test 
environment  The  maneuvers  have  already  been  shown  to  be  valuable  in  simulation,  but  they 
need  in-flight  validation  to  determine  if  they  can  be  used  during  flight  test  also.  In  particular, 
attributes  such  as  repeatability,  difficulty  to  set  up,  measurability,  and  safety  mred  to  be 
evaluated  before  these  maneuvers  can  be  used  confidently.  Some  work  has  already  been  dwic 
through  Air  Force  Test  Pilot  School  projects  and  generally  favorable  results  have  been 
obtained.  These  projects  were  limited  to  relatively  low  AOA  because  of  the  aircraft  available 
for  testing.  It  would  be  beneficial  to  fly  the  high  AOA  evaluation  maneuvers  on  the  NASA 
F-18  HARV,  F-16  MATV,  NASA  F-15  S/MTD  ACTIVE,  or  X-31  because  of  their  high  AOA 
capabilities. 

It  would  also  be  interesting  to  gather  lessons  learned  from  early  flight  test  programs  and 
determine  if  the  STEMS  maneuvers  would  have  been  able  to  detect  deficiencies  that  were 
missed  during  design.  If  cases  of  PIO.  roll  ratcheting,  oi  other  deficiencies  were  uncovered 
during  a  flight  test,  it  would  be  beneficial  to  have  an  evaluation  maneuver  capable  of 
uncovering  those  deficiencies  for  future  designs.  If  none  of  the  current  STEMS  maneuvers  can 
isolate  the  problem,  then  it  would  be  valuable  to  develop  a  new  maneuver  to  expose  that 
deficiency  and  include  it  in  STEMS. 
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Appendix  A 

Additional  Potential  Maneuvers 


An  enonnous  n'loiber  of  potential  uaneuvei^  were  considered  during  this  contract  but  tmly 
a  few  were  developed  and  tested  in  simulation.  This  appendix  includes  preliniinaiy  maneuvers 
from  Phase  I  tliat  were  not  tested  during  sLoulatfon  because  of  time  ermstnunts.  Some  of  these 
maneuvers  may  be  valuable  to  further  dcvelt^  and  add  to  STEMS.  However,  some  of  tlicse 
maneuvers  may  duplicate  other  STEMS  maneuvers  <k  may  not  work  well  fw  evaluatitMi,  so 
they  should  be  reviewed  and  compared  to  the  current  STEMS  maneuvers  before  being 
developed  during  simulation.  I  hey  are  include  in  this  appendix  as  a  source  of  ideas  for  future 
development  of  STEMS. 
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Grouping  of  n;.‘/ieuvo’^  95, 1S5.  243. 
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